1/4 


AD-A120  885  CURRENT  METHODOLOGIES  FOR  THE  ANALYSIS  OF  CONTINGENCV 
TABLES:  ROBUSTNESS  WITH  RESPECT  TO  SMALL  EXPECTED 
VALUES(U)  ARMY  MILITARY  PERSONNEL  CENTER  ALEXANDRIA  VA 
UNCLASSIFIED  R  A  XOLB  09  JUN  82  F/G  12/1  NL 


MICROCOPY  RESOLUTION  TEST  CHART 


MICROCOPY  RESOLUTION  TEST  CHART 
notional  aunuu  or  otanomoo-imi-a 


BK  FILE  COPY  A  120805 


CURRENT  METHODOLOGIES  FOR  THE  ANALYSIS  OF  CONTINGENCY 
TABLES:  ROBUSTNESS  WITH  RESPECT  TO  SMALL  EXPECTED  VALUES 
Major  Rickey  Arthur  Kolb 


Pinal  Report  9  June  1982 


I 

I 


Approved  for  public  release:  distribution  unlimited 


A  thesis  submitted  to  Georgia  Institute  of  Technology,  Atlanta,  Georgia 
in  partial  fulfillment  of  the  requirements  for  the  degree  of  Doctor  of 
Philosophy  in  the  School  of  Industrial  and  Systems  Engineering 


e 


82  10  28  051 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  Dmtm  Bnffd) 


REPORT  DOCUMENTATION  PAGE 


4.  TITLE  *an< Subtitle) 

Current  Methodologies  for  the  Analysis  of  Contin¬ 
gency  Tables:  Robustness  with  Respect  to  Small 
Expected  Values 


7.  author**; 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


3.  RECIPIENT'S  CATALOG  NUMBER 


S.  TYPE  OF  REPORT  A  PERIOD  COVERED 

Final  9  June  1982 


6.  PERFORMING  ORG.  REPORT  NUMBER 


B.  CONTRACT  OR  GRANT  NUMBER**; 


Major  Rickey  Arthur  Kolb 


9.  PERFORMING  ORGANIZATION  NAME  AND  ADDRESS 

Student,  HQDA,  MILPERCEN  (DAPC-OPP-E) 
200  Stovall  Street 
Alexandria,  VA  22332 


II.  CONTROLLING  OFFICE  NAME  AND  ADDRESS 

HQDA,  MILPERCEN,  ATTN:  DAPC-OPP-E, 
200  Stoyall  Street 
Alexandria,  VA  22332 


10.  PROGRAM  ELEMENT.  PROJECT,  TASK 
AREA  4  WORK  UNIT  NUMBERS 


12.  REPORT  DATE 


9  June  1982 


13.  NUMBER  OF  PAGES 

342 


I 


MONITORING  AGENCY  NAME  B  ADDRESS*;*  different  Non  Controlling  Otllce)  I  IS.  SECURITY  CLASS.  *of  this  report) 


IS*.  DECLASSIFICATION/ DOWNGRADING 
SCHEDULE 


S.  DISTRIBUTION  STATEMENT  (at  thla  Report) 


Approved  for  public  release;  distribution  unlimited 


17.  DISTRIBUTION  STATEMENT  (at  tft*  abstract  antovad  In  Black  30,  It  dllterent  tram  Raport) 


IS.  SUPPLEMENTARY  NOTES 


Thesis 

Georgia  Institute  of  Technology 


19.  KEY  WORDS  (Continue  an  ravaraa  aide  It  nacaaaarjr  and  Identity  by  black  number; 

Contingency  Tables,  Pearson  Chi-Square,  Kullback  Minimum  Discrimination 
Information,  Grizzle,  Starmer  and  Koch  (GSK)  Statistic,  Small  Expected  Values, 
Multinomial  Sampling,  Log-Linear  Model,  Minimum  Cell  Expectation,  Critical 
Expected  Value  Distributions. 


20.  ABSTRACT  (tSmtbaua  am  raearaa  at* a  H  naaaaaaty  and  Identity  by  block  number} 

The  primary  purpose  of  this  study  is  to  investigate  the  robustness  characteris¬ 
tics  of  some  of  the  current  methodologies  for  the  analysis  of  contingency 
tables  as  the  $ize  of  the  table  increases .  A  Monte  Carlo  simulation  provides 
estimates  of  exact  significance  levels  for  comparison  with  nominal  levels. 

An  extensive  array  of  tables,  probability  designs,  and  sample  sizes  are  used.- 


sornoM  op  *  mov  u  is  i 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  fBfcrn,  Data  Bntarad) 


SSCUWTV  CLASSIFICATION  Of  THI»  FAOSfWAaa  Bata  Matfd) 


Three  asymptotic  chi -squared  statistics  are  evaluated:  the  Pearson  statistic, 

* 

the  Kullback  minimum  discrimination  information  statistic,  and  the  Grizzle, 

S termer,  and  Koch  (GSK)  Wald  statistic.  A  95#  confidence  interval  criterion 

about  the  nominal  levels  is  defined,  and  certain  minimum  sample  sizes  (N  )  are 

m 

determined.  These  N  provide  the  parameters  for  calculation  of  critical  expect- 

in 

ed  value  (CEV)  distributions.  The  CEV  distributions  are  extensively  analyzed . 


The  GSK  statistic  is  shown  to  be  highly  conservative,  significantly  biased  by 
the  presence  of  zero  cells,  and  less  robust  as  the  size  of  the  table  increases. 
The  Kullback  statistic  is  shown  to  be  generally  liberal  with  best  performance 
at  the  lower  nominal  levels,  and  little  change  in  robustness  characteristics 

“l  >  ‘  V  ■> 

is  noted.  The  Pearson  statistic  is  shown  to  be  somewhat  conservative  and 
clearly  superior  to  the  other  statistics,  and  the  robustness  with  respect  to 
small  expected  values  is  significantly  improved  as  the  size  of , the  table 
increases .  ^ _ _ 


SICUfttTY  CLASSIFICATION  of  THIS  FAOK(iniM  Bata  KnlaratQ 


CURRENT  METHODOLOGIES  FOR  THE  ANALYSIS  OF  CONTINGENCY 
TABLES:  ROBUSTNESS  WITH  RESPECT  TO  SMALL  EXPECTED 


VALUES 


A  THESIS 


Presented  to 


The  Faculty  of  the  Division  of  Graduate  Studies 


Rickey  Arthur  Kolb 


In  Partial  Fulfillment 


of  the  Requirements  for  the  Degree 
Doctor  of  Philosophy  in  the  School  of  Industrial 
and  Systems  Engineering 


h 


Georgia  Institute  of  Technology 
Copyright  ©  1982  by  Rickey  Arthur  Kolb 


RRENT  METHODOLOGIES  FOR  THE  ANALYSIS  OF  CONTINGENCY 
TABLES:  ROBUSTNESS  WITH  RESPECT  TO  SMALL  EXPECTED 

VALUES 


ill 


ACKNOWLEDGMENTS 


No  one  can  complete  a  Fh.D.  without  the  assistance,  guidance, 
and  encouragement  of  many  people.  I  wish  to  thank  Dr.  William  W. 

Hines  who  first  encouraged  me  to  pursue  a  Ph.D.  degree.  Likewise,  I 
am  grateful  to  Col.  Jack  M.  Pollin,  Head  of  the  Department  of 
Mathematics,  USMA,  for  his  encouragement  and  allowing  me  to  return 
to  Georgia  Tech  to  begin  this  research. 

I  wish  to  thank  my  principal  advisor  Dr.  Douglas  C.  Montgomery 
for  his  continuing  guidance  and  assistance  throughout  this  research 
and  Dr.  James  W.  Walker  who  made  significant  contributions  and 
listened  patiently  to  my  ideas  at  each  step  of  the  research.  Also, 

I  wish  to  thank  Dr.  Harry  M.  Wadsworth  for  his  timely  suggestions 
and  the  other  members  of  my  reading  comnlttee.  Dr.  Russell  G.  Heikes 
and  Dr.  John  S.  Carson  III. 

No  Ph.D.  candidate  could  pursue  his  research  without  the 
assistance  of  the  library  staff.  I  wish  to  give  a  special  thanks  to 
Mrs.  Celeste  B.  Sproul  as  representative  of  the  many  library  personnel 
who  have  helped  me  at  Georgia  Tech,  West  Point,  and  Ft.  Leavenworth. 

Lastly,  I  wish  to  thank  my  wife,  Mary,  whose  constant  encourage¬ 
ment,  support,  and  understanding  made  it  possible  for  me  to  devote 
those  extra  concentrated  hours  needed  to  complete  this  research. 

Besides  that,  she  explained  to  our  four  sons,  Richard,  Patrick,  Robert, 
and  Edward,  why  their  Daddy  was  still  in  school. 


TABLE  OF  CONTENTS 


Page 


ACKNOWLEDGMENTS .  iii 

LIST  OF  TABLES .  vii 

LIST  OF  ILLUSTRATIONS .  viii 

SUMMARY . ix 

Chapter 

I.  INTRODUCTION  .  1 

1.1  Contingency  Tables  .  .  2 

1.2  Nature  of  the  Problem .  6 

1.3  Regression  Example  .  . .  9 

1.4  Scope . 19 

II.  HISTORICAL  OVERVIEW  AND  LITERATURE  REVIEW  .  21 

2.1  Methodologies  .  . .  21 

2.2  Small  Expected  Values  .  29 

2.3  Review  of  Previous  Studies . 31 

2.3.1  Lewontln  and  Felsensteln  (1965) .  31 

2.3.2  Haynam  and  Leone  (1965) .  32 

2.3.3  Craddock  (1966) .  33 

2.3.4  Suglura  and  Otake  (1965) .  34 

2.3.5  Odoroff  (1970) .  35 

2.3.6  Craddock  and  Flood  (1970) .  36 

2.3.7  March  (1970) .  37 

2.3.8  Yarnold  (1970) .  38 

2.3.9  Roscoe  and  Byars  (1971)  ............  39 

2.3.10  Margolin  and  Light  (1974)  .  40 

2.3.11  Camllli  and  Hopkins  (1978)  41 

2.3.12  Larntz  (1978)  ...  .  42 

2.3.13  Miller  (1979)  .  43 

2.3.14  Wang  (1979) .  44 

2.3.15  Cox  and  Plackett  (1980) .  45 

2.3.16  Discussion  of  Limitations  .....  .  46 

III.  CURRENT  METHODOLOGIES . 53 

3.1  Sampling  Models  .......  .  53 

3.2  Log-Linear  Model  . . 56 


Chapter  Page 

3.3  Hypothesis  Tests  .  62 

3.4  Maximum  Likelihood  Estimates .  80 

3.5  Pearson  Chi-Square  (X^) .  87 

3.6  Minimum  Discrimination  Information  (MDI)  .  . .  89 

3.6.1  General  Derivation  .  89 

3.6.2  The  Internal  Constraints  Problem  (ICP)  ....  93 

3.6.3  The  MDI  Log-Linear  Model  and  Hypothesis 

Tests  .  . .  96 

3.6.4  Asymptotic  Covariances .  108 

3.6.5  Asymptotic  Comparisons .  115 

3.7  Grizzle,  Starmer,  and  Koch  (GSK) .  118 

3.7.1  General  Derivation  .  119 

3.7.2  Log-Linear  Model  and  Hypothesis  Tests  .  121 

IV.  EXPERIMENTAL  DESIGN .  130 

4.1  Selection  of  Parameters .  130 

4.1.1  Exact  Levels  of  Significance  .  131 

4.1.2  Zero  Cells  and  Zero  Marginals .  134 

4.1.3  Probability  Designs  .  139 

4.1.4  Sample  Sizes .  148 

4.2  The  Exact  2*2  Program .  150 

4.3  The  Simulation  Program .  155 

4.3.1  Program  Design .  156 

4.3.2  Number  of  Tables .  160 

4.3.3  Program  Validation .  161 

V.  DATA  ANALYSIS .  168 

5.1  Basic  Data . 169 

5.1.1  Comparisons  with  Previous  Studies  .  169 

5.1.2  Analysis .  174 

5.2  Minimum  Cell  Expectation  (MCE) .  178 

5.2.1  Previous  Results  .  178 

5.2.2  Tabulation  of  Minimum  and  Maximum  Significance 

Levels  for  Minimum  Cell  Expectation 

Intervals  (MCEI) . 180 

5.2.3  Analysis  . .  180 

5.3  Critical  Expected  Value  (CEV)  Distributions  .  182 

5.3.1  Minimum  Sample  Size  (Nm)  .  182 

5.3.2  Calculation  of  CEV  Distributions .  191 

5.3.3  Analysis .  192 

5.4  y 2  and  Other  Parameters . 200 

VI.  CONCLUSIONS  AND  RECOMMENDATIONS .  203 

6.1  Conclusions . 203 

6.2  Recommendations  for  Future  Research  .  206 


APPENDIX  Page 

A.  APPROXIMATION  TO  MDI  STATISTICS .  208 

B.  PROBABILITY  DESIGNS  .  219 

C.  SAMPLE  SIZE  DESIGN .  228 

D.  EXACT  2x2  PROGRAM .  229 

E.  MONTE  CARLO  2x3  PROGRAM .  234 

F.  EXACT  2x2  DATA . ' .  238 

G.  MONTE  CARLO  2x2  DATA  (OVERLAP) .  242 

H.  MONTE  CARLO  2x3  DATA .  245 

I.  MINIMUM  AND  MAXIMUM  SIGNIFICANCE  LEVELS  FOR  MINIMUM 

CELL  EXPECTATION  INTERVALS  (MCEI)  .  250 

J.  MINIMUM  SAMPLE  SIZES  (N  ) .  259 

m 

K.  CRITICAL  EXPECTED  VALUE  (CEV)  DISTRIBUTIONS  ......  268 

BIBLIOGRAPHY  .  332 


n 


hi 


LIST  OF  TABLES 


Data  tot  Regression  Example 


Contingency  Table  Tests  of  Regression  Example 


h 

■  Sf 

'■.'it’1 

•  -? 

A 


Summary  of  Previous  Studies 


Degrees  of  Freedom:  r  *  s  Table 


Hypothesis  Tests  sad  Specified  Marginals  .  .  .  .  ,  '.  .  .  .  142 

2*2  Total  said  Unique  Number  of  Tables  .  . .  .  .  151 

2x2  Symmetric  Classifications  (H  divisible  by  4)  ....  .  153 


•93  Accuracy  Levels  (E) 


Monte  Carlo  Validation  (I)  ................  164 

Monte  Carlo  Validation  (II)  ................  165 


Comparisons  with 


tad  Leone  Data 


Comparisons  with  Craddock  Data  .......  .  171 


Comparisons  with  Craddock  and  Flood  Data 


Comparisons  with  Miller  Data  . . . .  173 


hi 


Conditional  Ism  Cell  Probabilities 


Extreme  Vector  Example  ..............  191 


Maximum  Two-way  Tables 


LIST  OF  ILLUSTRATIONS 


2  x  3  Table  of  Observations . 

Orientations  for  the  2x2x3  Table . 

Sets  of  Marginals  for  the  2x2x3  Table . 

2x2  Table  of  Probabilities . . . 

2x2  Table:  Product-Binomial  Sampling  .  .  .  .  .  .  . 

2x2  Table:  Homogeneity  Hypothesis . 

2x2x2  Table:  Fixed  One-Way  Marginals  (x,  )  .  .  . 

2x2  Table:  Multinomial  Sampling . 

2x2  Table:  Independence  Hypothesis  Reparameterized 

2x2  Table:  One  Zero  Marginal  (x#1  ■  0) . 

2x3  Table:  One  Zero  Marginal  (x  ^  *  0) . 

2x2  Table:  Standard  Arrangement . 

2x2  Table:  Interchanged  Categories . .  . 

Kullback,  2x3,  Probability  Vector  No.  4 . 

Kullback,  2x3,  Probability  Vector  No.  15 . 

Pearson,  2x3,  Probability  Vector  No.  15 . 

GSK,  2x3,  Probability  Vector  No.  15  .  .  . . 

95%  Confidence  Interval  Template  . 


SUMMARY 


Current  methodologies  for  the  analysis  of  contingency  tables 
are  being  used  on  larger  and  larger  tables  with  little  concern  for 
the  classic  problem  of  small  expected  values.  The  inference  is  that 
for  these  larger  tables  these  methodologies  are  more  robust  with 
respect  to  small  expected  values.  Previous  studies  of  this  small 
expected  value  problem  have  been  limited  as  to  the  statistics  investi¬ 
gated,  the  hypotheses  considered,  and  the  assessment  criteria  and 
table  designs  used.  There  has  been  little  evidence  of  robustness 
trends  as  the  table  size  increases. 

The  primary  purpose  of  this  study  is  to  investigate  the  robust¬ 
ness  characteristics  of  some  of  the  current  methodologies  for  the 
analysis  of'  contingency  tables  as  the  size  of  the  table  increases.  A 
Monte  Carlo  simulation  provides  estimates  of  exact  significance  levels 
for  comparison  with  nominal  levels .  An  extensive  array  of  tables , 
probability  designs,  and  sample  sizes  are  used.  The  underlying 
sampling  model  is  multinomial.  The  log-linear  model  is  used  to  define 
the  variable  relationships  and  provide  independence  hypothesis  test 
parameters.  Three  asymptotic  chi-squared  statistics  are  evaluated: 
the  Pearson  statistic,  the  Kullback  minimum  discrimination  information 
statistic,  and  the  Grizzle,  Starmer,  and  Koch  (GSK)  Wald  statistic. 

A  minimum  cell  expectation  (MCE)  criterion  is  used  to  show  that 
the  performance  of  the  statistics  generally  improves  as  the  MCE  in- 


creases.  The  second  moment  cf  the  probability  structure  is  shown  to 

be  a  useful  parameter  for  measuring  the  performance  of  each  statistic 

A  95%  confidence  interval  criterion  about  the  nominal  levels  is 

defined,  and  certain  minimum  sample  sizes  (N  )  are  determined  for 

m 

each  table,  statistic,  probability  design,  and  nominal  level  combina¬ 
tion.  These  provide  the  parameters  for  calculation  of  critical 
expected  value  (CEV)  distributions.  The  CEV  distributions  are  ex¬ 
tensively  analyzed. 

The  GSK  statistic  is  shown  to  be  highly  conservative,  signifi¬ 
cantly  biased  by  the  presence  of  zero  cells,  and  less  robust  as  the 
size  of  the  table  increases.  The  Kullback  statistic  is  shown  to  be 
generally  liberal  with  best  performance  at  the  lower  nominal  levels, 
and  little  change  in  robustness  characteristics  is  noted.  The 
Pearson  statistic  is  shown  to  be  somewhat  conservative  and  clearly 
superior  to  the  other  statistics,  and  the  robustness  with  respect  to 
small  expected  values  is  significantly  improved  as  the  size  of  the 


table  increases 


CHAPTER  I 


INTRODUCTION 

The  study  of  cross-classified  categorical  data  in  the  form  of 
contingency  tables  has  resulted  in  a  wealth  of  papers,  books,  and 
monographs  presenting  several  competing  methodologies.  These  metho¬ 
dologies  have  been  developed  through  an  evolutionary  process  with 
contributions  from  the  finest  statisticians  of  the  time.  Although 
these  methodologies  are  competing,  many  of  their  assumptions,  tech¬ 
niques,  and  problems  are  similar.  Each  decade  seems  to  bring  a  new 
set  of  ideas,  applications,  and  discussions  related  to  the  use  of 
these  methodologies  even  before  previous  problems  have  been  solved. 

The  focus  of  the  present  study  is  on  the  analysis  of  some  of  the 
current  methodologies  with  respect  to  a  classic  problem,  that  of 
small  expected  values. 

In  the  past  few  years  there  has  been  a  trend  toward  the  analysis 
of  larger  and  larger  contingency  tables  (more  variables  and/or  more 
categories  per  variable).  The  justification  for  the  use  of  the  current 
methodologies  (which  use  "approximate"  statistics  and  depend  on  asymp¬ 
totic  theory)  in  the  analysis  of  these  large  tables  has  been  based  on 
large  sample  sizes.  Often,  even  with  large  sample  sizes,  larger  con¬ 
tingency  tables  present  some  expected  cell  values  which  are  relatively 
small.  Historically,  statisticians  have  warned  against  the  use  of  these 
statistics  under  the  presence  of  small  expected  cell  values.  Yet,  there 


seems  to  be  little  concern  for  these  small  values  in  large  tables. 

The  inference  is  that  the  problems  associated  with  the  use  of  these 
statistics  become  less  significant  as  the  size  of  the  table  increases. 

With  the  extensive  use  of  the  current  methodologies,  particular¬ 
ly  for  large  contingency  tables,  there  seems  to  be  a  pressing  need  for 
an  investigation  of  the  robustness  of  these  methodologies  with  respect 
to  small  expected  values.  Of  primary  interest  is  the  examination  of 
trends  as  the  table  size  increases. 

1.1  Contingency  Tables 

A  contingency  table  is  a  presentation  of  count  data  resulting 
from  cross-classifications.  The  cross-classifications  are  variables, 
factors,  or  responses  which  have  a  number  of  levels  or  categories. 

Terms  used  synonymously  for  this  type  of  data  are  cross-classified, 
cross- tabulated,  categorical,  qualitative,  or  frequency  data.  These 
data  are  the  result  of  cross-classifying  a  population,  or  sample  from 
a  population,  and  accumulating  totals  for  each  "cell"  of  the  contin¬ 
gency  table.  A  cell  total,  then,  is  the  number  of  observations  from 
the  population  or  sample  that  fall  into  the  variable,  categorical 
combination  represented  by  that  cell.  The  table  summarizes  informa¬ 
tion  for  the  entire  population  or  sample,  where  every  observation  is 
categorized  into  one  and  only  one  cell. 

A  two-dimensional  (two-way),  r x s  contingency  table  has  two  vari¬ 


ables:  one  variable  having  r  categories  and  one  variable  having  s 
categories.  The  "complete"  cross-classification  gives  a  total  of  r»s 
cells.  The  following  notation  for  a  two-way,  r*s  table  will  be  used: 


{x^j}  =  table  of  observed  values; 

{p^j}  =  table  of  cell  probabilities; 

{m^j }  =  table  of  expected  values; 

s 

£  x..  -  x.  =  observed  row  marginals,  1-1, 2,..., s; 
j-1 

r 

][  x  .  -  x  .  =  observed  coluoen  marginals,  j-l,2,...,r; 

1-1 

r  s 

1  £  x. .  -  x  -  N  =  total  sample  size  or  population. 

i-1  j-1 

The  marginal  probabilities  (p^ »P. and  marginal  expected  values 
(mla,m  are  similarly  defined.  Estimates  of  the  probabilities  and 
expected  values  will  be  represented  with  the  usual  hat  notation  (i.e., 
p^j  and  n_^) .  This  notation  is  easily  extended  to  higher-way  tables 
(tables  with  more  than  two  variables)  simply  by  adding  more  subscripts 
Contingency  table  data  is  commonly  presented  in  tabular  form. 

For  a  2  x  3  table.  Figure  1  is  the  usual  presentation  for  the  table  of 
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Figure  1.  2x3  Table  of  Observations 
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observations.  Here  there  are  two  variables:  variable  A  with  two  cate- 
gories  and  variable  B  with  three  categories.  The  marginals  and  total 
sample  (or  population)  size  are  conveniently  displayed.  Similar  tables 
can  be  presented  for  the  probabilities  and  expected  values  as  well  as 
their  corresponding  estimates.  The  presentation  of  the  data  in  tabular 
form  becomes  more  difficult  as  the  number  of  variables  increases.  For 
example,  in  the  2x2x3  table  the  tabular  form  can  be  used  with  three 
different  orientations  as  given  in  Figure  2a.  Each  orientation  allows 
for  easy  calculation  and  presentation  of  a  different  combination  of  sets 
of  "two-way"  marginals  and  a  set  of  "one-way"  marginals.  Two-way 
marginals  are  sums  over  all  variables  except  two,  and  one-way  marginals 
are  sums  over  all  variables  except  one.  For  the  three  orientations 
displayed  in  Figure  2a,  the  sets  of  marginals  presented  are  given  in 
Figure  2b. 


one-way  marginals 

two-way  marginals 

(1) 

(xll«’x12*,x21*,x22*) 

^X1«1,X1*2,X1»3,X2»1,X2*2,X2*3) 

(2) 

0c.l.’*.2.) 

(Xll»  »x12* ,x21» ,x22«^ 

(x.ll»x.12’x.13’x»21*x*22»x»23) 

(3) 

^X1,1'X1*2,X1*3,X2*1’X2*2'X2«3^ 

(X.11»X.12»X.13,X.21’X*22,X.23) 

Figure  2b.  Sets  of  Marginals  for  the  2x2x3  Table 


An  alternative,  and  sometimes  more  convenient,  method  of  presen¬ 


tation  of  the  data  from  a  contingency  table  is  in  vector  form,  using 
lexicographic  order  on  the  subscripts  (from  left  to  right).  For  the 
2  *  3  table  the  vector  of  observations  would  be 

(xll,x12»x13,x21,x22,x23)* 
and  for  the  2x2x3  table  the  vector  would  be 


(xlll,x112*x113*x121*x122*x123,x211*x212»x213,x221,x222*x223)* 


This  fora  will  be  used  extensively  in  this  thesis  for  the  presentation 
of  cell  probabilities.  The  term,  probability  vector,  will  represent 
a  table  of  cell  probabilities. 

1.2  Nature  of  the  Problem 

The  term,  "small  expected  value",  is  meaningless  without  some 
knowledge  of  the  context  of  its  use.  With  reference  to  contingency 
table  analysis,  an  expected  value  is  "small"  if  it  causes  a  loss  of 
confidence  in  the  use  of  a  particular  methodology.  This  may  depend  on 
many  factors,  such  as  the  fora  of  the  sampling  distribution,  the  assumed 
model,  the  hypothesis  to  be  tested,  the  test  statistic,  or  the  nature 
of  the  results  desired.  Unfortunately,  for  contingency  table  analysis 
little  thoretical  or  empirical  evidence  is  available  to  know  when  an 
expected  value  is  really  "small".  Certain  "rules  of  thumb"  and  a 
historical  discussion  will  be  presented  in  Chapter  II. 


In  Che  analysis  of  contingency  Cables  Che  primacy  interest  is 
Co  investigate  the  relationships  among  the  variables.  Often,  a  model 
is  first  assumed,  then  hypothesis  tests  are  performed  using  statistics 
and  observed  data.  In  classical  hypothesis  testing  situations  the 
null  hypothesis  is  assumed  true,  and  the  distribution  of  a  statistic 
is  used  to  determine  whether  or  not  it  is  reasonable  to  have  obtained 
the  realization  of  the  statistic  as  calculated  from  the  observed  data. 

In  some  cases,  such  as  testing  the  mean  of  a  normal  distribution,  the 
form  of  the  distribution  is  known.  In  other  cases,  such  as  testing  the 
mean  of  an  unknown  distribution,  certain  "large  sample"  properties  of 
the  statistic  and  an  asymptotic  distribution  are  used  to  test  the  hypo¬ 
thesis.  In  still  other  cases,  such  as  regression  analysis  and  analysis 
of  variance,  a  model  is  first  assumed.  The  model  is  then  "fit"  based  on 
the  observations,  and  with  a  distribution  assumption,  hypothesis  tests 
are  performed  on  the  model. 

The  three  situations  differ  as  to  the  nature  of  the  assumptions 
and  the  errors  that  can  occur  when  there  are  departures  from  these 
assumptions.  In  the  first  case,  with  known  distributions  there  are  no 
assumptions,  and  the  test  is  exact  in  the  sense  that  probabilities 
associated  with  the  hypothesis  test  errors  can  be  calculated  exactly. 

In  the  second  case,  using  large  sample  properties  and  asymptotic  distri¬ 
butions,  uncontrolled  errors  occur  because  of  the  approximate  nature  of 
the  distributions  for  finite  samples.  In  the  performance  of  these 
hypothesis  tests  an  understanding  of  the  robustness  properties  of  the 
associated  statistics,  with  their  approximate  distributions,  is  needed, 
particularly  when  sample  sizes  are  relatively  small.  In  the  third  case 


uncontrolled  errors  occur  when  the  assumed  distributions  are  not  exact. 
Here  an  understanding  of  the  robustness  properties  of  these  tests  is 
needed  when  there  are  departures  from  these  assumed  distributions. 

Contingency  table  analysis  incorporates  aspects  of  the  second 
and  third  cases  above.  As  in  the  third  case  a  model  is  usually  assumed. 
In  regression  analysis  and  in  analysis  of  variance  the  model  is  very 
significant  and  is  the  focus  of  the  procedure.  In  contingency  table 
analysis  hypotheses  can  be  formed  without  the  model,  but  the  model  does 
provide  a  convenient  way  of  describing  the  relationships  among  the  vari¬ 
ables  and  the  hypothesis  tests.  Unlike  regression  analysis  and  analysis 
of  variance  no  distribution,  related  to  the  model,  is  assumed.  In 
fact,  a  "complete"  model  for  a  contingency  table  will  be  exact.  Like 
the  second  case  above,  the  distributions  of  the  statistics  used  to 
test  the  hypotheses  or  model  parameters  are  approximated  by  their 
asymptotic  distributions.  Therefore,  as  in  the  second  case  an  under¬ 
standing  of  the  robustness  properties  of  these  tests  is  needed.  The 
performance  of  these  tests  depends  on  how  well  the  asymptotic  distribu¬ 
tions  fit  the  exact  distributions  of  the  statistics,  a  criterion 
dependent  on  many  factors  including  both  sample  size  and  the  size  of 
the  table. 

From  the  discussion  above,  the  term,  "small  expected  values",  in 
relationship  to  contingency  table  analysis  refers  to  the  fact  that 
these  asymptotic  distributions  may  not  well  approximate  the  exact  dis¬ 
tributions  of  the  test  statistics  when  expected  values  are  small. 

Hence,  robustness  properties  are  related  to  the  performance  of  these 
test  statistics  with  respect  to  small  expected  values. 
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1.3  Regression  Example 

The  following  regression  example  provides  some  Insight  Into  the 
two  major  aspects  of  the  problem  to  be  Investigated  In  this  study. 

First  Is  the  basic  problem  that  these  approximate  chi-square  statistics 
do  not  perform  well  under  the  presence  of  very  small  expected  values. 
Second  is  the  problem  that  just  increasing  the  size  (number  of  cells) 
of  a  table  may  not  Improve  the  performance  of  the  statistics.  In  fact, 
there  may  even  be  a  degradation  in  performance. 

Consider  the  individual  height  and  weight  data  given  in  Table  1. 

Table  1.  Data  for  Regression  Example 


Weight  (lbs.) 

153 

175 

104 

135  ! 

177 

157 

124 

145 

186 

Height  (ins.) 

70 

72 

65 

68 

74 

72 

70 

67 

69 

73 

Assume  the  following  simple  linear  regression  model: 

y  ■  a  +  @x  +  e, 

where  weight  (y)  is  the  dependent  variable,  height  (x)  is  the  independent 
variable,  and  £  is  a  random  variable  representing  error.  Assume  that  the 
expected  value  of  the  error  is  zero,  that  the  variance  is  unknown  but  the 
same  for  each  value  of  x,  and  that  the  errors  are  uncorrelated.  Using 
the  least  squares  procedure  to  fit  the  model,  the  estimate  is 


y  -  -573.27  +  10.26x 


The  estimate  of  the  correlation  coefficient  (p)  is 

0  -  0.993. 

With  the  additional  assumption  that  the  errors  are  normally  distributed 
a  hypothesis  test  can  be  performed  on  the  linear  relationship  of  the 
variables;  i.e,,  H^:  8  *  0  or  p  ■  0.  Under  the  assumptions  and  null 
hypothesis  the  statistic  has  a  F  distribution  with  one  and  eight 
degrees  of  freedom.  At  the  .01  level  of  significance  F  ,  0  -  11.26. 

• wl )1)0 

The  realization  of  the  statistic  based  on  the  data  is  FQ  *  5732.05. 

As  might  be  expected  just  from  the  value  of  the  estimate  of  the  corre¬ 
lation  coefficient,  the  variables  are  highly  linearly  dependent. 

Suppose  the  data  in  Table  1  -are  now  categorized  and  placed  into 
a  contingency  table,  and  an  independence  hypothesis  test  is  performed. 
First,  appropriate  categories  must  be  selected.  For  a  2x2  table 
let  the  weight  categories  be  <150  and  ^150  and  the  height  categories 
be  <70  and  W0.  The  2x2  table  is 

Height 
<70  >70 

4 
6 

e 

4  6  10 

One  of  the  statistics  used  to  test  the  hypothesis  of  independence  is 
the  Pearson  chi-square. 


x  -  I  l  (x..  -  m  )  /m  , 
i-1  j-1  13  13  13 


where  Che  expected  values  (m^)  are  estimated  under  the  null  hypothesis 
of  independence.  In  particular,  the  maximum  likelihood  estimates  are 


-  v-j*'  1-1-2!  J'1-2- 

For  the  table  above,  these  estimates  are 

4 
6 

4  6  10 

2 

The  value  of  the  Pearson  statistic  is  X  *  10.00.  The  statistic  has  ■ 
an  asymptotic  chi-squared  distribution  with  one  degree  of  freedom. 
Rejection  values  of  the  statistic  corresponding  to  .10,  .05,  and  .01 
levels  of  significance  for  the  test  are  2.71,  3.84,  and  6.63,  respective 
ly.  Therefore,  the  hypothesis  of  Independence  is  rejected  for  as  low 
as  the  .01  level  of  significance.  This  result  agrees  with  the 
regression  test. 

Suppose  that  the  size  of  the  table  is  now  increased  to  a  2  *  3 
table  by  classifying  the  heights  into  the  three  categories  <68, 

(68,  72),  and  >72.  The  table  of  observations  is 


and  the  corresponding  maximum  likelihood  estimates  are 


4 

6 


The  value  of  the  Pearson  statistic  is  7.22.  The  statistic  now  has  an 
asymptotic  chi-squared  distribution  with  two  degrees  of  freedom.  The 
.10,  .05,  and  .01  rejection  values  are  4.61,  5.99,  and  9.21,  respec¬ 
tively.  The  independence  hypothesis  is  no  longer  rejected  at  the  .01 
level  of  significance,  but  the  hypothesis  is  still  rejected  at  the 
.05  and  .10  levels. 

Increasing  the  table  size  to  2x4  using  four  categories  of 


1.2 

1.2 

1.6 

1.8 

1.8 

2.4 

3  3  4 


height,  the  observed  table  is 
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and  the  corresponding  maximum  likelihood  estimates  are 


.8 

KB 

.8 

.8 

D 

KB 

KB 

1.2 

The  value  of  the  Pearson  statistic  is  5.83,  and  the  asymptotic  chi- 
square  statistic  with  three  degrees  of  freedom  has  .10,  .05,  and  .01 
rejection  values  of  6.25,  7.81,  and  11.34,  respectively.  The  inde¬ 
pendence  hypothesis  cannot  be  rejected  at  even  the  .10  level.  There 
appears  to  be  a  trend  as  the  size  (number  of  cells)  of  the  table 
increases . 

Before  discussing  this  "apparent"  trend,  consider  the  3x3 
table' where  the  weight  variable  is  divided  into  three  categories. 

The  observed  table  is 


<68 

Height 

(68,72) 

>72 

<145 

3 

1 

0 

Weight  (145,175] 

0 

2 

1 

>175 

0 

0 

3 

3 

3 

4 

and  the  corresponding  maximum  likelihood  estimates  are 


The  value  of  the  Pearson  statistic  is  11.11  and  can  be  compared  against 
four  degree  of  freedom  values  of  the  chi-squares  statistic  at  the  .10, 
.04,  and  .01  levels  of  7.78,  9.49,  and  13.28,  respectively.  The 
observed  value  of  the  statistic  again  calls  for  a  rejection  of  the 
null  hypothesis  of  independence  at  the  .10  and  .05  levels  but  not 
at  the  .01  level. 

Table  2  summarizes  the  four  contingency  table  tests  above,  along 
with  an  additional  test  based  on  the  most  extreme  table.  The  mos“ 
extreme  table  is  formed  by  defining  a  separate  category  for 
observed  height  and  weight.  For  the  data  in  Table  1,  this  results  in 
a  10  x 8  table.  The  Pearson  statistic  can  easily  be  calculated  with  a 
formula  given  by  Cramer  (1946,  pg.  443), 


X  /N(q-l)  <  1, 


where  q  ■  min(r,s)  for  an  r  *  s  table.  The  value  of  one  is  obtained 
when  all  the  rows  or  columns  contain  one  and  only  one  entry  different 


from  zero.  Then. 


Note  that  this  criterion  is  met  in  the  2x2  table, and  therefore, 

X2  -  10(2-1)  -  10.  For  the  extreme  10*8  table,  X2  -  10(8-1)  -  70. 

Referring  to  Table  2,  the  trend  from  the  2x2  through  the  2x4 
2 

tables  is  that  the  X  statistic  is  decreasing  and  becoming  less  sensi¬ 
tive  to  departures  from  the  null  independence  hypothesis.  The  conside- 

2 

ration  of  the  3  x  3  table  reverses  this  trend  since  the  X  statistic 

increases  and  again  rejects  the  hypothesis  at  the  .10  and  .05  levels. 

At  the  extreme  10x8  table  the  statistic  has  became  very  large,  70; 

but,  the  hypothesis  is  not  rejected  at  any  of  three  levels.  What  is 

interesting  about  this  example  is  that  the  data  remain  constant,  and 

the  variables,  based  on  the  regression  test,  seam  so  clearly  dependent. 

Why  are  there  such  discrepancies  in  these  hypothesis  tests?  The 

2 

answer  lies  in  the  nature  of  the  X  statistic  and  the  error  involved  in 

using  the  chi-squared  distribution  as  an  approximation  to  the  exact 

distribution  of  the  statistic. 

2 

The  X  statistic  has  an  exact,  discrete  distribution.  For  a 
given  sample  size  and  given  size  table  there  are  only  a  discrete  number 
of  possible  observed  tables.  The  chi-squared  distribution  is  contin¬ 
uous,  and  therefore  some  error  is  always  present.  For  a  two-way,  rxs 
contingency  table  the  number  of  degrees  of  freedom  associated  with  the 
Independence  hypothesis  test  is  (r-l)(s-l).  Basically,  the  number  of 
of  degrees  of  freedom  of  the  test  increases  as  the  size  of  the  table 
increases.  The  power  of  a  test  is  a  measure  of  the  sensitivity  of  the 
test  to  departures  from  the  null  hypothesis.  Normally,  in  hypothesis 
testing  situations  the  power  of  the  test  improves  as  the  degrees  of 
freedom  increase.  This  is  not  the  case  in  the  example  above.  In  fact, 
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just  the  opposite  is  true  from  the  2x2  through  the  2x4  tables. 

With  the  fixed  data,  as  the  size  of  the  table  increases,  the  table 
becomes  sparser  in  the  sense  that  the  average  expected  value  (m  *  N/k, 
where  k  is  the  number  of  cells)  decreases.  Table  2  lists  these  average 
expected  values  and  the  minimum  expected  values  for  each  table  of  the 
example.  From  the  first  three  tables  it  might  be  concluded  that  the 
power  of  the  test  decreases  as  the  table  becomes  sparser,  but  the  3x3 
table  would  invalidate  this  conclusion.  Note,  however,  that  the  mini¬ 
mum  cell  expectation  (MCE)  decreases  from  the  2  x  2  to  the  2x4  table 
and  then  increases  with  the  3X3  table.  Hence,  it  might  be  concluded 
that  the  power  decreases  as  the  MCE  decreases.  Whatever  the  case,  the 
power  of  the  test  is  clearly  affected  by  the  cell  expected  values.  The 

reason  for  this  is  that  the  chi-squared  distribution  is  not  performing 

2 

well  as  an  approximation  for  the  exact  distribution  of  the  X  statistic 

under  the  presence  of  small  expected  values. 

Before  leaving  this  example,  two  additional  comments  should  be 

made.  The  first  is  that  the  selection  of  the  boundary  points  for  the 

categorization  of  the  data  was  critical  to  the  results  of  the  hypothesis 

tests.  The  points  were  deliberately  chosen  to  demonstrate  certain 

aspects  of  the  small  expected  value  problem.  Selection  of  other  boundary 

points  could  lead  to  other  hypothesis  test  decisions.  However,  the 

extreme  table  is  unique.  Here  the  hypothesis  could  not  be  rejected  at 

any  of  the  three  levels  of  significance.  It  is  interesting  to  note 

that  as  long  as  one  of  the  variables  has  unique  data  points,  the 
2 

Pearson  X  statistic  can  be  calculated  from  Equation  (1-1),  and  it  will 
never  reject  the  null  hypothesis  at  the  .10  level  regardless  of  the  data, 


sample  size,  or  size  of  table.  The  optimum  selection  of  these  boundary 
points  is  another  problem  [e.g.,  see  Hamdan  (1968)].  In  most  contin¬ 
gency  table  applications  the  data  are  not  the  continuous-type  data  as 
in  the  previous  example.  Usually  the  data  are  true  count  data, 
resulting  from  the  cross-classification  of  a  sample  or  population,  and 
an  extreme  table  does  not  exist. 

The  second  comment  is  that  the  power  of  this  test  can,  in 
general,  be  improved  by  increasing  the  sample  size  without  changing 
the  size  of  the  table.  This  would  not  change  the  degrees  of  freedom 
of  the  test,  but  would  increase  the  average  expected  values  and,  in 
general,  increase  the  minimum  expected  values.  The  chi-squared  distri¬ 
bution  would  better  approximate  the  exact  distribution  of  the  statistic, 
and  the  power  of  the  test  would,  in  general,  increase.  In  the  previous 
example  suppose  the  same  data  were  exactly  replicated.  The  regression 
test  would  be  improved  (made  more  powerful)  since  its  "degrees  of 

freedom"  would  now  be  18  for  the  F-test  of  independence.  However,  while 
2 

the  X  values  in  Table  2  would  double,  the  degrees  of  freedom  of  the 
chi-square  tests  would  not  change.  These  tests  would  now  all  reject  the 
hypothesis  of  independence  at  all  levels  of  significance.  The  minimum 
expected  values  and  average  expected  values  would  all  double.  The 
power  of  the  tests  would  clearly  increase. 

This  example  demonstrates  several  significant  aspects  of  contin¬ 
gency  table  tests.  The  power  of  these  tests  is  related  to  how  well  the 
chi-squared  distribution  approximates  the  exact  distribution  of  the 
statistic  and  to  the  size  of  the  expected  values.  For  a  fixed  sample 
size  these  expected  values  decrease  as  the  size  of  the  table  increases. 
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and  in  spite  of  the  increase  of  the  degrees  of  freedom  of  the  test, 
the  power  can  still  decrease.  In  general,  as  the  sample  size  in¬ 
creases  the  power  of  the  test  increases  because,  with  an  increase  in 
the  expected  values,  the  chi-squared  distribution  better  approximates 
the  exact  distribution  of  the  statistic.  Finally,  the  discrete  nature 
of  the  statistic  causes  perturbations,  making  it  difficult  to  observe 
any  specific  trends. 

1.4  Scope 

This  study  focuses  on  the  most  popular  statistics  currently  in 
use  to  test  independence  hypotheses  in  contingency  tables.  The  primary 
purpose  of  this  study  is  to  investigate  the  robustness  characteristics 
of  these  statistics  with  respect  to  small  expected  values  as  the  size 
of  the  table  increases.  A  secondary  purpose  is  to  compare  the  perfor¬ 
mance  of  these  statistics  and  make  recommendations  for  their  use.  The 
main  vehicle  for  the  study  is  a  Monte  Carlo  simulation.  The  simulation 
provides  estimates  of  exact  levels  of  significance  to  compare  with 
specified  nominal  levels.  Trends  are  discussed  and  conclusions  drawn 
with  respect  to  the  stated  purposes. 

Chapter  II  presents  a  historical  overview  of  the  development  of 
contingency  table  analysis,  the  current  statistics,  and  the  small  ex¬ 
pected  value  problem.  Also,  previous  numerical  and  empirical  studes  are 
extensively  reviewed.  A  convenient  table  summarizes  these  studies,  and 
a  final  subsection  discusses  their  limitations  and  restrictive  scope 
with  respect  to  the  small  expected  value  problem. 

Chapter  III  provides  a  detailed  presentation  of  the  current  metho¬ 
dologies  for  the  analysis  of  contingency  tables.  This  chapter  begins 


'  •  **»-' 


Muiiaa 


with  a  discussion  of  the  underlying  sampling  models  and  the  assumed 
log- linear  model  and  relates  them  to  the  independence  hypotheses.  Maxi¬ 
mum  likelihood  estimates  are  then  derived.  Next,  the  three  current, 
competing  statistics  are  presented.  Finally,  some  comparisons  are  made 
showing  the  asymptotic  equivalence  of  these  statistics. 

Chapter  IV  describes  the  experimental  design.  This  includes  dis¬ 
cussions  of  the  selection  of  parameters  and  the  designs  of  the  Monte 
Carlo  and  exact  programs.  Here,  justifications  are  given  for  the 
selection  of  the  primary  level  of  significance  parameter,  the  probability 
design  parameters,  the  sample  sizes,  and  the  number  of  tables  to  be 
generated  by  the  Monte  Carlo  simulation.  The  problem  of  zero  cells  and 
zero  marginals  is  also  discussed.  The  last  subsection  provides  a  vali¬ 
dation  of  the  Monte  Carlo  simulation  using  exact  results. 

Chapter  V  presents  the  major  results  of  the  study  through  the 
analysis  of  the  data.  The  chapter  begins  with  a  presentation  and  analy¬ 
sis  of  the  basic  data  from  the  Monte  Carolo  simulation  and  the  exact 
algorithm.  Comparisons  are  made  with  results  from  other  studies. 
Following  a  procedure  used  in  a  previous  study,  minimum  and  maximum 
significance  levels  for  minimum  cell  expectation  intervals  (MCEI)  are 
reported  and  analyzed.  Minimum  sample  sizes  are  then  derived  for  a 
specified  criterion,  and  critical  expected  value  (CEV)  distributions  are 
calculated.  These  CEV  distributions  are  extensively  analyzed  and  trends 
noted. 

Chapter  VI  lists  the  significant  conclusions  of  the  study,  pro¬ 
vides  recommendations  for  the  use  of  the  statistics  Investigated,  and 
makes  recommendations  for  future  research. 
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CHAPTER  II 

HISTORICAL  OVERVIEW  AMD  LITERATURE  REVIEW 

This  chapter  presents  a  historical  overview  of  contingency 
table  analysis  with  particular  Cephas is  on  the  current  Methodologies 
to  be  investigated  and  the  small  expected  value  problem.  Also,  pre¬ 
vious  studies  related  to  this  problea  are  extensively  reviewed. 

2.1  Methodologies 

Methodologies  for  the  analysis  of  contingency  tables  have,  to 
varying  degrees,  differed  in  several  ways: 

(1)  sampling  model 

(2)  method  of  estimation  of  expected  values 

(3)  assumed  model  of  the  parameters 

(A)  method  for  selecting  the  appropriate  model 

(5)  interpretation  of  the  model  selected 

(6)  test  statistics 

(7)  considerations  of  "factors”  and  "responses". 

A  review  of  each  of  these  topics  will  not  be  undertaken.  This  review 

will  concentrate  on  (2),  (3),  (A),  and  (6)  which  lead  to  the  specific 

differences  of  the  most  popular  competing  methodologies  today. 

Most  statisticians  agree  [e.g.,  see  Cochran  (1952)  and  Bishop, 

Fienberg,  and  Holland  (1975)]  that  the  classic  paper  by  Pearson  (1900b) 

2 

was  the  first  to  propose  a  useful  statistic,  X  (Pearson's  chi-square 


goodness-of-fit) ,  to  use  in  the  analysis  of  contingency  tables.  In 

this  paper  Pearson  established  the  necessary  distribution  theory  for 

finding  significance  levels  when  expected  values  can  be  calculated  under 

2 

a  null  hypothesis.  The  X  statistic, 


compares  the  observed  values  (x^)  with  these  expected  values  (m^)  based 

on  the  chi-squared  distribution.  Fisher  (1924)  outlined  a  proof  of 

2 

this  limiting  chi-squared  distribution  for  X  when  the  parameters  are 
estimated  from  the  observations.  A  complete  proof  was  given  by  Cramer 
(1946)  and  later,  under  more  general  conditions,  by  Birch  (1964). 

Pearson  (1900a)  used  the  test  specifically  for  a  2*2  table  and  later 
[Pearson  (1904)]  for  an  r * s  table. 

Pearson  (1900b)  assumed  an  underlying  multivariate  normal  distri¬ 
bution  for  the  data  in  all  cases,  even  when  the  data  was  from  a  discrete 
source  such  as  categorical  variables.  In  contrast.  Yule  (1900)  proposed 
considering  the  levels  (categories)  of  the  variables  as  fixed.  He 
investigated  the  relationship  of  these  variables  by  considering  a  cross- 
product  ratio  for  the  2x2  table, 


mllm22 

m12m21 


where  the  m^  are  the  expected  values.  He  looked  at  several  functions 
of  this  cross-product  ratio,  using  them  to  compare  various  tables  with 
known  structures.  For  the  next  25  years  an  argument  between  Yule  and 


Pearson  over  these  competing  approaches  dominated  the  literature  on 
contingency  tables  [e.g.,  see  Pearson  and  Heron  (1913)  and  Fisher 
(1922a, 1924) ] .  Pearson's  goodness-of-fit  statistic,  X  ,  has  a  multi¬ 
tude  of  applications  [see  Lancaster  (1969a)]  and  is  one  of  the  most 
used  statistics  today.  Pearson's  underlying  continuous  model  for  cate¬ 
gorical  data  has  been  essentially  disregarded,  except  for  some  use  by 
Lancaster  (1949,  1950,  1957,  1969a)  in  a  partitioning  of  chi-square 
method.  Yule's  concept  of  considering  the  categories  as  fixed,  with 
some  modifications,  is  the  historical  beginning  of  today's  categorical 
data  models. 

Both  Yule  and  Pearson  considered  only  two-dimensional  (variable) 
structures.  It  was  not  until  Bartlett  (1935)  that  a  higher-way  (three 
or  more  variables)  table  was  investigated.  Bartlett  analyzed  a  2  *  2  x 2 
table  and  extended  Yule's  cross-product  ratio  to  define  the  concept  of 
second-order  interaction.  He  also  proposed  a  chi-square  test  for  the 
presence  of  this  interaction  based  on  the  following  sum  of  squares 
statistic: 

8  ,  .  8 

£  x/a  »x  7  1/a, 

r  L.  r’ 

r*l  r-1 

where  x  is  the  total  deviation  from  the  expectation  (a^)  in  each  cell. 

2 

The  similarity  to  Pearson's  X  is  obvious.  This  classic,  concise  paper 

also  presented  several  new  ideas:  it  linked  the  term  "independence”  to 

"no  interaction",  offered  an  exact  test  when  one  of  the  cell  expected 

k 

values  was  small,  suggested  an  extension  of  the  procedure  to  2  tables. 
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and  demonstrated  the  problems  associated  with  multi-way  tables  having 
variables  with  more  than  two  levels  (categories) . 

Roy  and  Kastenbaum  (1956)  further  extended  Yule’s  theory  and 
Bartlett's  higher-order  interactions  and  tests.  They  introduced  a 
"multiplicative”  model  which  generalized  Bartlett’s  second-order  inter¬ 
action  (which  they  called  "three-factor  interaction")  from  2x2x2 
tables  to  r  x g  x  t  tables.  Birch  (1963)’  showed  that  these  interactions 
could  be  defined  as  linear  combinations  of  the  logarithms  of  the  expected 
frequencies.  He  introduced  a  special  parameterization  and  demonstrated 
the  relationship  between  the  values  of  these  parameters  and  the  corre¬ 
spondence  of  the  observed  and  expected  values.  This  formed  the  basis 
for  what  is  known  as  the  log-linear  model.  This  model  provides  the 
flexibility  to  analyze  the  structure  of  any  contingency  table,  in  a 
manner  similar  to  analysis  of  variance  and  stepwise  regression,  using 
a  number  of  different  methodologies.  The  log-linear  model  has  been 
extensively  described  by  Bishop,  Fienberg,  and  Holland  (1975)  with  sup¬ 
porting  mathematical  proofs  given  by  Haberman  (1970,  1973,  1974)  and 
contributions  in  the  use  of  the  model  by  Bishop  (1967,  1969a, b),  Goodman 
(1968,  1970,  1971a, b,  1973),  Fienberg  (1968,  1970a, b,  1977,  1979), 

Grizzle  and  Williams  (1972),  and  many  others. 

Other  models  have  been  suggested,  including  an  additive  model 
[Bhapkar  and  Koch  (1968)],  the  Lancaster  (1949,  1950,  1969a)  partition¬ 
ing  model,  and  a  general  linear  model  [Nelder  and  Wedderburn  (1972)  and 
Nelder  (1974)]  with  the  log-linear  model  as  a  special  case.  The 
additive  model  has  been  used  in  special  situations  such  as  sample  survey 
data,  drug  comparisons,  and  biological  assays  [e.g.,  see  Johnson  and 
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Koch  (1970)  and  Koch  and  Reinfurt  (1971)].  Johnson  and  Koch  (1970) 
discuss  the  advantage  of  the  additive  model  for  sample  survey  data. 

In  general,  the  log-linear  model  is  the  most  extensively  used,  pro¬ 
viding  convenient  parameters  for  most  hypothesis  testing  situations. 

An  excellent  discussion  and  comparison  of  the  corresponding  additive 
and  multiplicative  interaction  terms  for  these  two  models  is  given  by 
Darroch  (1974) . 

The  current,  competing  methodologies  use  the  log-linear  model,  and 
primarily  differ  in  the  method  of  estimating  expected  values  and  in  the 
test  statistics  used  to  select  appropriate  parameters  for  the  log-linear 
model  (hypothesis  testing).  The  three  most  popular  methods  of  estima¬ 
tion  are  maximum  likelihood,  minimum  modified  chi-square,  and  minimum 

discrimination  information.  The  four  most  popular  test  statistics  are 
2 

the  Pearson  X  ,  the  minimum  modified  chi-square,  the  minimum  discriminar- 
tion  information,  and  the  likelihood  (log-likelihood)  ratio. 

Maximum  likelihood  estimates  are  based  on  the  well-known  theory 
presented  by  Fisher.  Although  he  makes  no  reference  to  maximum  likeli¬ 
hood,  Bartlett  (1935)  used  these  estimates  for  his  expected  values 
(a^)  in  a  2x2x2  contingency  table.  Bartlett  provided  a  cubic  equa¬ 
tion,  given  to  him  by  Fisher,  to  solve  for  these  expected  values.  The 
equation  cannot  be  solved  in  closed  form,  but  only  through  trial  and 
error  or  some  numerical  iteration  technique.  It  wasn't  until  Demlng 
and  Stephen  (1940)  that  a  general  iterative  scheme  was  presented  to 
solve  for  the  maximum  likelihood  estimates  in  a  three-way  table.  Brown 
(1959)  gave  a  first  proof,  although  somewhat  incomplete,  on  the  conver¬ 
gence  of  the  procedure.  Ireland  and  Kullback  (1968)  offered  a  complete 


proof.  Other  proofs  were  given  by  Fienberg  (1970b),  Darroch  and 
Ratcliff  (1972),  and  Haberaan  (1970,  1973,  1974a).  Haberman  (1972) 
presented  an  excellent  computer  program  of  the  procedure  for  a  general 
N-way  table.  As  contingency  table  analysis  evolved,  Roy  and  Kastenbaum 
(1956),  Lewis  (1962),  Birch  (1963),  Goodman  (1963,  1964,  1968,  1971a,b, 
1973),  Bishop  (1967,  1969),  and  others  have  used  these  estimates. 

Maximum  likelihood  estimates  were  well-known  to  have  desirable 
asymptotic  properties  such  as  consistency  and  efficiency.  Neym&n 
(1949),  in  a  classic  paper,  defined  a  class  of  estimates,  best  asymp¬ 
totically  normal  (BAN),  which  included  the  maximum  likelihood  estimates. 
He  discussed  their  properties  in  detail.  Neyman  (1949)  went  on  to 
specifically  define  other  BAN  estimators,  in  particular,  a  minimum  modi¬ 
fied  chi-square  estimator  using  the  generalized  quadratic  theory 

established  by  Wald  (1943).  Along  with  these  estimates,  Neyman  proposed 

2 

a  corresponding  test  statistic  similar  to  the  Pearson  X  , 

Y2  ■  I  (xi-mi)2/xi, 

where  the  observed  values  now  divide  the  squared  differences  of  the 
observed  and  expected  values.  Bhapkar  (1966)  demonstrated  the  equiva¬ 
lence  of  this  statistic  and  the  Wald  statistic  for  linear  hypotheses 
and,  with  a  linearization  technique,  for  nonlinear  hypotheses.  Grizzle, 
Starmer,  and  Koch  (1969)  generalized  this  theory  and  established  a  pro¬ 
cedure  (GSK)  based  on  weighted  least  squares.  The  GSK  procedure  has 
been  applied  to  a  series  of  different  kinds  of  contingency  tables 
using  linear  and  log-linear  models  [e.g.,  Koch  and  Reinfurt  (1971), 


Johnson  and  Koch  (1971),  and  Grizzle  and  Williams  (1972)].  Today, 
due  to  the  extensive  and  varied  use  of  the  GSK  procedure  and  the 
availability  of  excellent  computer  programs  [e.g.,  Landis  et.al. 

(1976)],  this  technique  has  become  very  popular. 

The  third  approach  is  closely  related  to  maximum  likelihood  and 
is  based  on  the  theory  of  minimum  discrimination  information  (MDI)  as 
presented  by  Kullback  (1959) .  This  theory  was  first  applied  to  con¬ 
tingency  tables  by  Kullback,  Kupperman,  and  Ku  (1962).  Later  applica¬ 
tions  and  expansion  of  the  theory  were  given  by  Ireland  and  Kullback 
(1968),  Ku  and  Kullback  (1968,  1974),  Ireland,  Ku,and  Kullback  (1969), 

Ku,  Varner,  and  Kullback  (1971),  Gokhale  (1972),  Berkson  (1972),  and 
many  others.  The  theory  and  procedures,  together  with  excellent  examples 
and  computer  programs,  have  been  presented  in  a  book  by  Gokhale  and 
Kullback  (1978a).  Estimates  are  obtained  using  iterative  techniques 
[see  Ku  and  Kullback  (1968)]  in  a  manner  similar  to  those  for  maximum 
likelihood  estimates.  In  fact,  for  independence  hypotheses  these  esti¬ 
mates  are  identical  to  the  maximum  likelihood  estimates  [see  Gokhale  and 
Kullback  (1978a,  pg.  3)  and  Meyer  (1980)].  For  independence  hypotheses 
the  test  statistic  is  the  minimum  discrimination  information  statistic 
(MDIS) , 

21  ■  2  ^  x1  £n(xi/mi) , 

where  the  above  MDI  estimates  for  the  expected  values  (m^)  provide  the 
minimum  value  for  this  statistic.  This  statistic  is  identical  to  the 
log-likelihood  ratio  statistic  first  proposed  by  Fisher  (1922b),  later 


used  for  independence  in  contingency  tables  by  Neyman  and  Pearson 
(1928) ,  and  extended  to  other  contingency  table  hypotheses  by  Wilkes 
(1935),  Woolf  (1957),  Chakravarti  and  Rao  (1959),  and  Gart  (1966). 

For  other  hypothesis  tests,  such  as  homogeneity,  the  MDI  estimates 
are  not  maximum  likelihood,  and  the  MDIS  is  not  log-likelihood  ratio 
[e.g.,  see  Gokhale  and  Kullback  (1978b)].  Probably  the  most  interest¬ 
ing  aspect  of  the  MDI/MDIS  approach  is  that,  as  first  shown  by 
Kullback  (1959),  the  log-linear  model  is  a  natural  result  of  the 
theoretical  derivation  of  the  minimization  of  the  statistic. 

All  the  methods  previously  discussed  have  been  approximate, 
dependent  on  asymptotic  distributions  and  requiring  data  to  be 
"relatively"  large.  Fisher  (1934)  first  proposed  an  "exact"  test  for 
2*2  tables.  Actually,  Yates  (1934)  first  published  the  test  but  gave 
recognition  to  Fisher.  Irwin  (1935)  independently  presented  the  same 
procedure.  The  Fisher  (Fisher-Irwin  or  Fisher-Yates)  exact  test  consid¬ 
ers  the  margins  of  the  observed  table  as  fixed  and  calculates  the  hyper- 
geometric  probability  of  the  observed  table  and  all  tables  farther  from 
the  "no  interaction"  hypothesis.  The  total  sum  of  these  probabilities 
is  then  compared  to  a  given  level  of  significance.  Freeman  and  Halton 
(1951)  applied  the  test  to  a  general  r*s  table,  giving  appropriate 
formulas.  Computer  programs  for  r  *s  tables  were  developed  by  March 
(1972)  and  Schwartz  (1972),  Shaffer  (1972)  presented  a  technique  to 
extend  the  Fisher  test  to  higher-way  (three  or  more  variables)  tables. 
Other  similar  "exact"  tests  have  been  suggested  by  Ha.  une  (1937,  1940), 
Finney  (1948),  Tocher  (1950),  Leslie  (1955),  Boschloo  (1970),  Zahn  and 
Roberts  (1971),  and  Agrestl  and  Wackerly  (1977).  In  Fisher  and  Irwin's 


time,  due  to  the  calculations  required,  the  exact  test  was  only  useful 
for  very  small  tables  (N£20).  Even  today,  with  the  availability  of 
high-speed  computers  the  usefulness  of  the  test  is  restricted.  Guide¬ 
lines  for  use  of  the  exact  test  for  r*s  tables  on  an  IBM  360  computer 
have  been  given  by  Agresti,  Wackerly,  and  Boyett  (1979),  and  a  more 
efficient  algorithm  has  recently  been  presented  by  Pagano  and  Halvorsen 
(1981).  Exact  procedures  are  no  panacea.  They  have  frequently  been 
attacked  simply  on  the  basis  of  the  assumption  of  fixed  margins  [e.g., 
see  Camilli  and  Hopkins  (1978)  or  Berkson  (1978)].  Due  to  their  com¬ 
putational  difficulties,  these  procedures  have  seen  little  use  in 
practice.  They  have  primarily  been  used  to  validate  or  compare  the 
approximate  methods. 

2.2  Small  Expected  Values 

All  three  of  the  approximate  methods  discussed  ate  based  on  the 
BAN  estimators  described  by  Neyman  (1949)  and  are  asymptotically  equiva¬ 
lent.  The  asymptotic  distribution  for  each  statistic  is  the  chi-squared. 
The  inaccuracy  of  chi-square  tests  for  small  samples  was  well-rknown 
to  Pearson,  Fisher,  Yule,  and  Yates  in  the  early  development  of  contin¬ 
gency  table  analysis.  In  practice,  as  stated  by  Yates  (1934),  it  was 
customary  to  regard  the  Pearson  chi-square  tests  as  "sufficiently" 
accurate  if  no  cell  had  an  expected  value  less  tna*.  five.  No  empirical 
or  theoretical  evidence  had  been  presented  to  verify  this  magic  "rule 
of  thumb".  Even  today,  most  statistical  textbooks  give  the  five  "rule 


of  thumb"  for  chi-square  tests  for  any  goodness-of-fit  or  contingency 
table  hypothesis.  However,  a  brief  survey  of  the  literature  reveals  as 
many  as  nine  other  "rules  of  thumb". 


Cochran  (19S2)  presented  a  detailed  discussion  of  the  develop- 

2 

ment  of  the  chi-square  goodness-of-fit  test  (Pearson's  X  )  along  with 

his  classic  summary  of  recommendations  for  the  use  of  Fisher's  exact 
2 

test,  Pearson's  X  test,  Haldane's  (1937,  1940)  exact  mean  and  variance 

2 

calculations,  and  Yates'  (1934)  continuity  correction  to  Pearson's  X  . 
These  recommendations  were  based,  as  Cochran  states,  more  on  "experi¬ 
ence"  than  anything  else.  They  did  provide,  at  the  very  least,  a 
"straw-man"  for  statisticians  to  evaluate.  With  the  exception  of  the 
five  "rule  of  thumb",  these  guidelines  have  been  discarded  due  to 
problems  associated  with  Yates'  continuity  correction  [e.g.,  see 
Conover  (1974)  or  Haber  (1980)],  current  use  of  the  log-linear  model 
and  associated  statistical  techniques,  and  the  availability  of  high¬ 
speed  computers. 

Some  investigations  have  been  made  with  regard  to  the  properties 
of  current  procedures  when  expected  values  are  small.  Most  of  these 
have  been  empirical,  either  numerical  comparisons  or  Monte  Carlo  studies 
Haberman  (1977)  presented  the  only  recent  theoretical  proofs  providing 
some  insight  into  the  asymptotic  properties  of  statistics  for  large 
contingency  tables  with  small  expected  frequencies.  His  results  are 
related  to  the  log-linear  model.  Conditions  are  provided  for  the  asymp¬ 
totic  normality  of  linear  functionals  of  log-mean  vectors  (maximum  like¬ 
lihood  estimates)  and  for  the  asymptotic  chi-squared  distribution  of  the 
2 

Pearson  X  and  the  log-likelihood  ratio  statistics.  Essentially,  these 
conditions  will  allow  some  cells  to  have  fixed  small  expected  values 
while  asymptotic  properties  are  maintained.  These  results  hold  for  any 
size  table.  Apart  from  this,  little  theoretical  information  is  avail- 


able  concerning  the  effect  of  small  expected  values  on  large  tables. 

2.3  Review  of  Previous  Studies 

Most  of  the  empirical  work  related  to  chi-square  tests  have 
2 

focused  on  the  Pearson  X  as  a  goodness-of-f it  test  for  the  multinomial. 
Of  particular  note  is  Yarnold's  (1970)  extensive  investigation  which 
provided  significant  insight  into  the  small  expected  value  properties 
of  this  test.  The  first  empirical  studies  related  to  contingency  tables 
although  somewhat  limited  in  scope,  provided  some  interesting  results 
and  ideas  for  follow-on  studies.  The  studies  of  Odoroff  (1970)  and 
Craddock  and  Flood  (1970)  are  the  most  frequently  discussed.  However, 
several  other  studies  appeared  earlier,  including  Monte  Carlo  studies 
by  Lewontln  and  Felsenstein  (196S)  and  Craddock  (1966),  an  "exact" 
study  by  Hayman  and  Leone  (1965),  and  a  numerical  comparison  study  by 
Suglura  and  Otake  (1968) .  These  studies  along  with  follow-on  studies 
will  be  reviewed  in  approximate  chronological  order.  A  summary  chart 
is  provided  in  Table  3  and  a  discussion  of  the  limitations  of  these 
studies  is  presented  in  subsection  2.3.16.  Only  one  of  the  studies 
[Camilli  and  Hopkins  (1978)3  that  is  related  soley  to  continuity  correc¬ 
tions  in  2 x 2  tables  will  be  reviewed.  The  results  of  this  study  are 
representative  of  the  results  of  other  similar  studies  [e.g.,  Plackett 
(1964),  Grizzle  (1967),  Mantel  and  Greenhouse  (1968)  and  Conover  (1974)] 
In  a  recent  study  Haber  (1980)  has  provided  some  different  conclusions 
concerning  "alternative"  (to  Yates)  continuity  corrections. 

2.3.1  Lewontln  and  Felsenstein  (1965) 

Lewontln  and  Felsenstein  presented  a  Monte  Carlo  investigation 
2 

of  the  Pearson  X  test  of  homogeneity  in  2*N  tables.  They  compared 


2 

the  estimated  exact  distributions  of  the  Pearson  X  statistic  with  the 

chi-squared  distribution  in  the  tail  areas,  specifically  at  certain 

nominal  values  (.01,  .02,  .05,  .10).  They  considered  27  different 

combinations  of  fixed  marginals  and  probability  arrangements  for  2x5 

and  2xio  tables.  Only  symmetric  N-column  marginals  with  variances  of 

0,  1.6,  and  3.6  were  considered.  They  classified  each  table  arrangement 

2 

as  to  how  well  the  distribution  of  the  Pearson  X  statistic  agreed  with 
the  chi-squared  distribution. 

2 

The  results  of  the  study  indicated  that  the  Pearson  X  statistic 

was  generally  conservative  for  five  or  more  degrees  of  freedom  and  more 

robust  with  respect  to  small  expected  values  than  usual  "rule  of  thumb" 

would  indicate.  Lewontln  and  Felsenstein  suggested  that  the  statistic 

could  be  used  as  a  test  of  homogeneity  in  2  x n  tables  when  expected 

values  were  as  low  as  one.  It  should  be  noted  that  the  sampling  plan 

used  for  their  Monte  Carlo  procedure  restricted  both  sets  of  marginals, 

which  implies  a  hypergeometric  sampling  model  rather  than  the  product- 

binomial  model  related  to  the  homogeneity  hypothesis  (see  Section  3.1). 

This  raises  some  doubt  as  to  the  general  validity  of  their  conclusions. 

An  interesting  aspect  of  this  study  was  the  calculation  of  esti- 

2 

mated  mean  and  variances  for  the  X  statistic  in  all  54  cases  considered. 
The  agreement  with  exact  formulas  given  by  Haldane  (1940)  was  excellent. 
2.3.2  Haynam  and  Leone  (1965) 

2 

Haynam  and  Leone  studied  the  Pearson  X  statistic  when  used  both 
as  a  goodness-of-fit  test  statistic  for  the  multinomial  and  as  an  inde¬ 
pendence  and  homogeneity  test  statistic  for  contingency  tables.  With 
regards  to  contingency  tables  their  study  investigated  the  exact  dis- 


tributions  of  X  under  the  three  sampling  conditions  of  no  margins  fixed 
(multinomial) ,  one  set  of  margins  fixed  (product-multincmial) ,  and  both 
sets  of  margins  fixed  (hypergeometric)  in  2x3  and  3x3  tables  with 
equiprobable  cells.  They  compared  these  exact  distributions  with  the 
chi-squared  distribution. 

Results  were  similar  for  all  three  sampling  models.  Haynam  and 

2 

Leone  concluded  that  the  distribution  of  the  X  statistic  is  fairly  well 

2 

approximated  by  the  X  distribution  in  the  region  up  to  0.9  cummulative 
probability,  but  tended  to  overestimate  "rather  badly”  exact  probabili¬ 
ties  in  the  tail  area.  Unfortunately,  for  hypothesis  testing  situations 
the  tail  areas  are  of  primary  importance. 

This  study  was  very  limited  by  the  consideration  of  only  the 
equiprobable  case  and  sample  sizes  of  ten  and  fifteen.  However,  as  the 
authors  mention,  the  amount  of  computer  time  needed  for  these  exact 
calculations  was  significant. 

2.3.3  Craddock  (1966) 

2 

Craddock  performed  a  Monte  Carlo  investigation  of  the  Pearson  X 

statistic  when  used  in  the  3x3  equiprobable  contingency  table  to 

measure  the  "evidence  of  association"  (test  for  independence).  Craddock 

generated  10,000  observations  of  tables  for  sample  sizes  (N)  of  10,  15, 

2 

20,  25,  30,  40,  50,  and  100.  He  then  calculated  the  Pearson  X  for  each 
table  and  formed  histograms  in  the  range  0  to  20  using  a  grouping  inter¬ 
val  of  0.2.  He  reported  estimated  percentile  values  for  these  histo¬ 
grams  for  the  important  percentiles  of  50,  90,  95,  98,  99,  and  99.9  and 
compared  these  to  the  chi-squared  distribution  values. 

Craddock’s  results  indicated  excellent  agreement  to  the  chi- 


squared  distribution  throughout  the  entire  range  from  N«100  down  to 
N“30.  For  lower  values  of  N  the  agreement  was  "spotty";  but,  Craddock 
suggested  that  at  the  important  hypothesis  testing  percentiles  the 
Pearson  statistic  could  be  used  with  these  3  *  3  tables  for  N  as  low  as 
10. 

2.3.4  Sugiura  and  Otake  (1968) 

Sugiura  and  Otake  used  numerical  evaluations  to  compare  various 
"improved"  tests  for  r  x  s  contingency  tables  with  small  frequencies  in 
some  cells.  The  test  statistics  considered  were: 

(1)  Fisher-Irwin  exact  test 

2 

(2)  Pearson  X  test  (maximum  likelihood  estimates) 

(3)  Yates'  continuity  correction 

(4)  Likelihood  Ratio  (LR)  test 

(5)  LR  test  with  Dandekar  modification  [Rao  (1952,  pg.  203)] 

(6)  LR  test  with  Yoshimura  (1963)  correction 

2 

(7)  Pearson  X  test  with  authors'  correction 

(8)  Method  I  by  Nass  (1959) 

(9)  Method  II  by  Nass  (1959) 

(10)  Method  I  by  Gart  (1966) 

(11)  Method  II  by  Gart  (1966) 

(12)  Modified  Dandekar  method  by  authors. 


The  authors  used  four  examples  for  the  study:  a  2x2  example  from  Rao 
(1952),  a  2x2  symmetric  case,  a  2x2  large  marginals  case,  and  a  2x3 
example  from  Yates  (1934).  They  used  the  exact  test  (1)  as  their 
criterion  for  the  2x2  tables  and,  for  the  2x3  table,  exact  tests 
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based  on  ordering  of  tables  by  the  Pearson  X  statistic  and  the  LR 
statistic. 

Sugiura  and  Otake's  study  Indicated  that  the  methods  (5),  (10), 
(11),  and  (12)  were  somewhat  better  than  the  others  for  2x2  tables, 
though  they  were  more  conservative  than  exact  levels.  Gart's  methods 
(10)  and  (11)  were  conservative  for  2x3  tables.  The  LR  modification 
by  Yoshimura  (6)  always  performed  better  than  the  LR  statistic  (4) . 

Nass's  methods  (8)  and  (9)  did  not  perform  very  well.  Other  methods 
gave  results  similar  to  (2). 

It  should  be  noted  that  most  of  these  tests  were  restricted  to 
two-way  tables,  and  some  required  one  variable  restricted  to  two  cate¬ 
gories.  The  results  of  this  study  are  very  limited.  Only  four  examples 
were  considered, and  the  exact  test  itself  is  known  to  be  conservative 
and  may  not  be  a  good  criterion  for  comparing  other  tests. 

2,3.5  Odoroff  (1970) 

Odoroff  presented  a  study  to  compare  the  small  sample  properties 

of  twelve  tests  for  interactioh  in  2x2x2  and  3*2x2  contingency 

tables.  The  twelve  tests  were  formed  by  combining  three  test  statistics 
2 

(Pearson  X  ,  minimum  logit  chi-square,  and  likelihood  ratio)  with  four 
estimation  procedures  [Iterative  maximum  likelihood  and  three  variations 
of  minimum  logit  chi-square  due  to  Gart  and  Zweifel  (1967)].  One  set  of 
two-way  marginals  were  fixed,  and  the  observations  were  considred  to  be 
sampled  from  rxs  independent  binomial  distributions.  The  results  were 
thus  limited  to  r  x  s x  2  tables  under  this  product-binomial  sampling 
scheme.  Odoroff  claimed  that  his  hypothesis  formulation  was  equivalent 
to  Bartlett's  (1935)  no  second-order  interaction.  An  evaluation  was  made 


mm 


as  to  the  adequacy  of  the  chi-squared  distribution  in  approximating 
exact  levels  of  significance  (nominal  levels  at  .05  and  .01)  under  the 
null  hypothesis.  Also,  power  calculations  were  performed  under  select¬ 
ed  alternatives  and  comparisons  made  as  to  computational  simplicity. 

For  the  2  x  2  *  2  tables  calculations  were  made  by  complete  enumeration, 
so  levels  were  exact.  For  3x2><2  tables  a  Monte  Carlo  method  was 
used  to  provide  estimates  of  exact  levels. 

Odoroff  generally  concluded  that  the  exact  levels  of  the  minimum 

2 

logit  chi-square  tests  and  the  Pearson  X  test  were  better  approximated 
by  the  chi-squared  distribution  than  were  the  exact  levels  of  the  like¬ 
lihood  ratio.  The  exact  levels  of  tests  using  the  minimum  logit  chi- 
square  estimation  were  better  approximated  than  the  exact  levels  based 
on  maximum  likelihood  estimation.  No  significant  power  differences  were 

observed.  The  non-iterative  minimum  logit  chi-square  estimations,  the 

2 

corresponding  test  statistic,  and  the  Pearson  X  were  the  easiest  to 
compute. 

An  interesting  aspect  of  Odoroff’ s  study  was  that  results  were 
presented  in  relation  to  a  special  computation  of  minimum  cell  expecta¬ 
tion  (MCE) .  Odoroff  used  these  MCE  to  ,,.ontrol  his  selection  of  cell  pro¬ 
babilities.  In  general,  the  statistics  performed  worse  for  lower  MCE. 
2.3.6  Craddock  and  Flood  (1970) 

Craddock  and  Flood  made  a  Monte  Carlo  investigation  of  the 
2 

Pearson  X  statistic  in  r  *  s  contingency  tables  from  3*2  to  5x5  with 


expected  values  under  the  null  hypothesis  of  independence  from  one  to 
five.  The  study  considered  only  equiprobable  tables,  so  the  expected 
values  were  equal  for  all  cells. 
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Craddock  and  Flood  developed  histograms  for  the  calculated 
2 

Pearson  X  and  provided  "smoothed”  tables  of  percentile  values  ranging 

from  one  to  99.9  percent.  They  demonstrated  the  convergence  of  the 

2 

distribution  of  the  Pearson  X  statistic  to  the  chi-squared  distribution 
throughout  its  range.  Focusing  on  the  5%  and  50%  percentiles,  they  pro¬ 
vided  curves  to  reflect  speeds  of  convergence.  They  concluded  that  the 
normal  five  "rule  of  thumb"  for  minimum  expected  values,  at  least  for 

these  equiprobable  tables,  was  too  conservative.  From  the  smoothed 

2 

histograms  they  observed  that  the  behavior  of  the  Pearson  X  statistic 
for  a  given  value  of  N  was  in  general  better  for  larger  tables.  This 
led  them  to  suggest  that  with  a  fixed  sample  size  analysts  should  con¬ 
sider  using  larger  contingency  tables  even  if  expected  values  are  below 
five. 

2.3.7  March  (1970) 

2 

March  performed  a  study  of  the  Pearson  X  statistic  in  2x3 

tables.  He  considered  all  possible  tables  for  sample  sizes  (N)  of  six 

through  30  and  36  and  42  where  the  expected  value  in  each  cell  is  greater 

than  or  equal  to  one.  He  compared  the  chi-square  probabilities  of  the 
2  2  2 

X  statistic  [i.e.,  P(x  >_  X  )]  and  the  exact  cumulative  hypergeometric 
probabilities  of  the  2><3  table  with  fixed  marginals.  These  probabili¬ 
ties  were  compared  in  terms  of  their  absolute  percentage  differences  and 
on  their  extent  of  agreement  in  certain  specified  exact  probability 
regions,  (.000-. 005),  (.005-. 010),  (.010-. 050),  (.050-. 100),  (.100-. 150), 
(.150-. 200),  and  (.200-1.00).  Also,  the  number  of  times  that  the  chi- 
squared  distribution  underestimated  and  overestimated  the  exact  proba¬ 


bilities  were  calculated. 


2 

March's  results  showed  that  the  Pearson  X  approximation  improves 

as  the  sample  size  (N)  increases  and,  hence,  as  the  average  expected 

value  (N/6)  increases.  As  a  general  approximation  to  the  exact  hyper- 

2 

geometric  distribution,  March  rated  the  X  statistic  as  poor.  However, 

March  noted  that  when  used  as  a  test  of  significance  (independence)  with 

2 

the  usual  nominal  levels  (e.g.,  .05),  the  X  statistic  gave  results  very 
similar  to  the  "exact"  test  even  for  relatively  low  expected  values 
(i.e.,  less  than  five). 

An  interesting  aspect  of  March's  study  was  the  running  accumula¬ 
tion  of  overestimates  and  underestimates.  March  generalized  that  the 
ratio  of  underestimates  to  overestimates  increased  as  the  exact  proba¬ 
bility  increased  and  decreased  as  the  sample  size  decreased. 

2.3.8  Yarnold  (1970) 

Yarnold  came  to  similar  conclusion  as  Craddock  and  Flood  and 

others  concerning  the  conservative  nature  of  the  five  "rule  of  thumb" 

2 

for  the  Pearson  X  statistic.  He  limited  his  empirical  investigation 

to  the  goodness-of-fit  hypothesis  of  a  multinomial  distribution. 

As  a  result  of  his  investigation,  Yarnold  provided  a  rule  for 

2 

guiding  the  use  of  the  Pearson  X  statistic  in  any  general  hypothesis 
testing  situation  where  it  is  appropriate.  Yarnold  states  that,  "If  a 
number  of  classes  s  is  three  or  more  and  if  r  denotes  the  number  of 
expectations  less  than  five,  then  the  minimum  expectation  may  be  as 
small  as  5r/s."  This  is  essentially  an  extension  of  Cochran's  (1952) 
recommendations  and  broadens  the  use  of  the  statistic  to  include  con¬ 
ditions  when  some  expected  values  are  less  than  five.  It  remains  to 
be  seen  if  the  rule  is  appropriate  for  contingency  table  tests. 


2.3.9  Roscoe  and  Byars  (1971) 


Roscoe  and  Byars  performed  a  Monte  Carlo  study  of  the  Pearson 

2 

X  test  for  goodness-of-fit  and  for  "independence"  in  a  "typical 
behavior  research  situation".  Only  the  later  test  will  be  discussed 
here.  The  "independence"  test  was  based  on  sampling  for  the  r * s 
tables  with  the  independent  row  marginals  fixed.  Thus  the  test  was 
actually  the  test  for  homogeneity  across  rows  (see  Section  3.3). 

Tables  varied  from  2*2  to  5*5.  Three  types  of  underlying  sampling 
distributions  were  used:  uniform,  moderately  skewed,  and  extremely 
skewed.  Fixed  sample  sizes  for  the  independent  rows  were  equal,  al¬ 
though  the  authors  did  comment  on  an  extension  to  the  reported  study 
allowing  for  unequal  sample  sizes. 

Roscoe  and  Byars  demonstrated  that  under  the  null  hypothesis  of 
homogeneity  when  sampling  from  a  uniform  distribution,  "reasonable" 
results  [using  Cochran’s  (1952)  criteria  for  exact  levels  of  significance 
as  compared  to  nominal  levels]  are  obtained  at  the  .05  nominal  level  with 
average  expected  values  as  low  as  two  and  at  the  .01  level  with  values 
as  low  as  four.  Their  results  further  Indicated  that  with  moderate 
departures  from  uniformity  expected  values  should  be  maintained  at 
four  for  the  .05  level  and  six  for  the  .01  level,  and  with  extreme  de 
partures  these  values  should  be  six  and  ten,  respectively. 

It  should  be  noted  that  these  recommendations  are  for  "average" 
expected  values,  and  under  departures  from  uniformity  they  allow  for 
some  relatively  small  expected  values  to  be  balanced  by  larger  ones. 
Roscoe  and  Byars  believed  that  this  approach  would  be  most  helpful 


since  most  studies  involve  a  uniform  or  equiprobable  distribution, 


and  comparisons  could  be  made. 
2.3.10  Margolin  and  Light  (1974) 


Margolin  and  Light  investigated  the  small  sample  behavior  of 
Pearson's  X^,  Kullback's  minimum  discrimination  information  statistic 
(21),  a  sample  version  of  Goodman  and  Kruskal's  lambda  (L),  and  their 
own  C  statistic.  They  considered  a  3x2  table  under  the  hypothesis 
of  homogeneity  where  the  marginals  x^  and  are  fixed,  and  limited 
their  examination  to  x^  ■  x^  *  N/2,  N  taking  values  of  five  and  10. 
They  used  11  probability  structures  for  the  marginal  pi# (i*l,2,3) . 

They  calculated  exact  tail  area  probabilities  (levels  of  significance) 
for  each  statistic  at  nominal  levels  of  .10,  .05,  .025, and  .01  by  com¬ 
pletely  enumerating  the  tables  for  each  structure  and  using  the  chi- 
squared  distribution  as  the  null  distribution.  A.  very  interesting 
aspect  of  this  study  Included  separately  considering  the  case  where 
marginal  observed  x^  were  zero.  Margolin  and  Light  used  the  chi- 
squared  distribution  with  one  degree  of  freedom  instead  of  two  for  these 
situations.  This  provided  both  exact  "conditional"  and  "unconditional" 
tail  area  probabilities. 

In  general,  the  results  of  this  study  indicated  that  in  these 

tail  areas  the  null  distribution  of  C  was  better  approximated  by  the 

2 

chi-squared  than  the  null  distribution  of  X  ,  and  both  were  considerably 

better  approximated  than  the  null  distribution  of  21.  L  was  so  erratic 

and  liberal  that  it  was  clearly  not  appropriate  for  use  as  a  test 

statistic  for  homogeneity.  Margolin  and  Light's  results  further 

2 

indicated  that  the  C  and  X  statistics  were  well  approximated  at  or 

2 

near  the  equlprobable  vector.  For  other  vectors  the  X  statistic  was 
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generally  conservative  and  tne  C  statistic  showed  no  pattern.  The 

2 

three  statistics  (X  ,  C,  21)  performed  much  better  under  the  "condi¬ 
tional"  criterion.  This  should  have  been  expected  since  there  is 
indeed  a  loss  of  degrees  of  freedom  under  the  presence  of  zero  margi¬ 
nals,  as  has  been  shown  by  many  authors  [e.g.,  see  Bishop,  Fienberg, 
and  Holland  (1975,  pp.  115-119)]. 

Margolin  and  Light  concluded  their  study  with  a  theorem  and 
2 

proof  relating  the  X  and  21  statistics  in  r  *  2  tables  where  x  ^  =* 

x  _  =  N/2.  The  relation  is 
•  l 

(2  £n  2)X2  ?_2I  >  X2. 

It  should  be  noted  that  Margolin  and  Light’s  selection  of 

Kullback's  "independence"  test  statistic  (21)  was  not  appropriate  for 

the  homogeneity  test  [e.g.,  see  Gokhale  and  Kullback  (1978b)]. 

2.3.11  Camilli  and  Hopkins  (1978) 

Carailli  and  Hopkins  presented  a  Monte  Carlo  study  of  the  ade- 
2 

quacy  of  the  Pearson  X  test,  with  and  without  Yates’  (1934)  correction 
for  continuity,  for  2x£  contingency  tables.  They  considered  the  three 
common  sampling  models:  (1)  both  marginals  fixed,  (2)  one  marginal 
fixed,  and  (3)  no  marginals  fixed.  Model  (1)  corresponds  to  Fisher's 
exact  test  probabilities  based  on  the  hypergeometric  distribution. 

Camilli  and  Hopkins  stated  that  this  distribution  was  seldom  seen  in 
actual  research  studies  and  concluded  that  there  was  a  serious  problem 
in  the  use  of  Fisher's  exact  test  since  the  data  usually  had  not  resulted 
from  fixed  marginals.  Model  (2)  was  the  model  used  by  Roscoe  and  Byars 
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(1971)  and  model  (3)  by  Craddock  and  Flood  (1970).  Cam! Hi  and  Hopkins 

generated  10,000  random  tables  using  specified  probability  structures 

under  models  (2)  and  (3)  and  estimated  exact  significance  levels  for 
2 

the  Pearson  X  ,  with  and  without  Yates'  correction,  at  the  nominal 
.01,  .05,  and  .10  levels. 

Camilli  and  Hopkins  concluded  that  the  tests  without  Yates'  cor¬ 
rection,  for  homogeneity  [model  (2)]  and'  independence  [model  (3)],  gave 
accurate,  though  somewhat  conservative,  type  I  error  probability  state¬ 
ments  when  the  sample  size  was  twenty  or  more,  even  when  the  expected 
cell  values  in  one  or  two  cells  were  as  low  as  one  or  two.  Yates' 
correction  was  shown  to  be  totally  Inadequate  under  models  (2)  and  (3), 
causing  the  statistic  to  be  ultra-conservative  with  a  corresponding  loss 
of  power.  This  last  conclusion  agrees  with  those  of  other  studies  [e.g., 
Conover  (1974)], 

2.3.12  Laratz  (1978) 

Larntz  considered  a  variety  of  categorical  data  models  with  and 

without  estimated  parameters  and  used  exact  and  Monte  Carlo  methods  to 

determine  exact  levels  of  significance  for  three  "goodness-of-fit" 

2  2 

statistics:  likelihood  ratio  G  ,  Pearson  X  ,  and  a  variant  of  the 
2 

Freeman-Tukey  T  .  He  tested  five  models: 

(1)  a  multinomial  model 

(2)  a  group  helping  model  in  a  3  *  2  table  with  fixed  column 
margins 

(3)  A  no-association  model  with  both  margins  fixed  (hypergeo- 
metrlc)  in  a  3x5  table 
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(4)  a  quasi-independence  model  in  a  5*5  table 

(5)  a  no  second-order  interaction  model  in  a  3  * 3  *  3  table. 

2  2 

Larntz's  conclusions  were  that  the  approximate  G  and  T  tests 

yielded  exact  levels  that  were  typically  in  excess  of  nominal  levels 

2 

with  moderate  expected  values  (1. 5-4.0)  and  that  the  Pearson  X  statis¬ 
tic  closely  attained  exact  levels  for  a  wide  range  of  sample  sizes, 
expected  values,  and  models.  These  results  with  regard  to  contingency 
tables  were  consistent  with  those  of  Odoroff  (1970)  and  Margolin  and 
Light  (1974)  for  G2  and  X2. 

2 

Larntz  provided  an  explanation  for  the  superiority  of  X  in  small 

samples.  He  calculated  the  minimum  contribution  to  each  statistic  for 

observed  cell  counts  of  zero  and  one  with  varying  cell  expectations. 

2 

The  Pearson  X  was  least  affected  by  the  small  observed  counts. 

2.3.13  Miller  (1979) 

2 

Miller  also  investigated  the  Pearson  X  and  the  likelihood  ratio 

2 

G  statistics  when  expected  values  were  small  and  parameters  were  esti¬ 
mated  from  the  data.  In  particular,  he  considered  tests  of  independence 

in  two-way  contingency  tables  (2*2,  5*2,  and  5*5)  when  one  or  more 

2  2 

of  the  expected  values  were  five  or  less.  He  evaluated  the  X  and  G 
statistics,  with  and  without  Yates  continuity  correction,  when  used  to 
test  both  independence  and  homogeneity.  Monte  Carlo  methods  were  used 
to  form  the  empirical  distributions  of  these  statistics  and  compare  them 
with  the  chi-squared  distribution  at  nominal  levels  .01,  .05,  and  .10. 

Miller's  results  indicated  that  the  Pearson  X  statistic  was 
robust  with  respect  to  small  expected  values.  It  performed  well  when 
expected  values  were  as  low  as  two  and  even  as  low  as  0.5  when  the 
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number  of  such  cells  was  small  with  respect  to  the  total  number  of  cells. 

2 

The  likelihood  ratio  G  statistic  usually  performed  well  for  minimum 

expected  values  of  five,  but  its  performance  was  Inconsistent  for  values 

less  than  five.  In  general,  Yates'  continuity  correction  gave  a  more 

conservative  test.  These  results  agreed  with  those  of  Margolin  and  Light 

(1974),  Camilli  and  Hopkins  (1978),  and  Larntz  (1978).  In  the  2*5  and 

5*5  tables.  Miller  noted  a  slightly  poorer  performance  of  the  Pearson 
2 

X  at  the  equiprobable  vector  when  expected  values  were  very  small.  He 
concluded  that  this  was  because  all  the  expected  values  were  small  with 
no  larger  values  for  balance.  Actually,  his  criteria  required  that 
sample  sizes  for  the  uniform  vectors  be  smaller,  and  this  alone  would 
probably  cause  a  poorer  performance. 

2.3.14  Wang  (1979) 

Wang  extended  Odoroff's  study  to  consider  tests  for  main  effects 
when  zero  interaction  was  assumed  in  the  manner  of  "partitioning  of 
conditional  independence"  described  by  Goodman  (1970).  As  in  Odoroff's 
study,  the  results  are  limited  to  r*s*2  tables  under  the  product- 
binomial  sampling  scheme.  Wang  used  eight  of  the  twelve  Odoroff  tests, 
excluding  those  associated  with  likelihood  ratio  which  performed  very 
poorly  for  Odoroff.  Wang  used  Odoroff's  Monte  Carlo  procedures  and  the 
same  2*2*2  and  3*2*2  tables.  The  eight  tests  were  evaluated  with 
respect  to  the  exact  conditional  levels  of  significance  (at  nominal  .05 
and  .01  levels)  in  rejecting  the  null  hypothesis  of  zero  column  (row) 
effect,  assuming  zero  row  by  column  effect  at  specified  nominal  levels 
of  rejection  (.10,  .25,  .50,  and  .75). 


From  among  the  eight  tests,  the  minimum  logit  statistic  with  one 
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of  the  minimum  logit  estimation  techniques  and  the  Pearson  X  with 
another  of  the  minimum  logit  estimation  techniques  were  identified  as 
best  with  respect  to  closeness  of  approximation  and  simplicity  of  com¬ 
putational  procedure.  The  first  test  was  the  same  best  test  found  by 
Odoroff  in  testing  for  independence.  Wang  used  the  same  minimum  rell 
expectation  (MCE)  computations  as  Odoroff  in  reporting  his  results. 

A  different  aspect  of  Wang's  study  was  the  inclusion  of  varied 
levels  for  rejection  of  the  independence  hypothesis.  In  general,  as 
the  MCE  increased,  lower  levels  for  rejection  of  the  independence  hypo¬ 
thesis  were  allowed. 

2.3.15  Cqx  and  Plackett  (1980) 

Cox  and  Plackett  suggested  three  ways  for  improving  current  asymp¬ 
totic  methods  for  analyzing  contingency  tables.  According  to  Cox  and 
Plackett,  these  methodologies  provide:  (i)  better  approximations  to 
moments  of  cell  frequencies  (expected  values)  than  those  derived  from 
the  iterative  scaling  procedure,  (ii)  exact  distributions  obtained  by 
enumerating  all  tables  consistent  with  given  marginal  totals,  and 
(ill)  simulated  random  sampling  for  tables  where  exact  enumeration  is 
impossible".  Their  3tudy  was  restricted  to  the  consideration  of  tables 
with  fixed  marginals  using  "exact"  distributions  as  a  criterion  for 
comparison. 

To  compare  their  approximations  in  (i),  Cox  and  Plackett  used 

If 

a  2  x  2  and  2x3  table  with  arbitrary  cross-product  ratios  and  a  2  (k«3) 
table  with  arbitrary  k-way  interaction.  Their  approximations  were 
clearly  superior  to  the  asymptotic  maximum  likelihood;  however,  these 


new  approximations  had  little  affect  on  the  asymptotic  methods  of 
inference . 


Using  several  examples  (3x3x3,  2  *  2  *  18,  2*  15,  2*2*  t), 

Cox  and  Flackett  demonstrated  that  complete  enumeration  or  Monte  Carlo 
methods  could  be  used  in  a  variety  of  contingency  table  cases.  They 
concluded  that  exact  and  Monte  Carlo  methods  could  be  used  with  small 
samples,  and  then  "avoid  the  mass  of  complicated  recommendations . . .". 

In  their  examples  Cox  and  Placket t  compared  the  "exact"  dis¬ 
tributions  of  the  Pearson  and  likelihood  ratio  with  the  chi- 

squared  distribution.  In  general,  their  study  agreed  with  those  of 
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others,  that  the  Pearson  X  was  closer  to  the  chi-squared  distribution 

2 

than  the  likelihood-ratio  G  . 

2.3.16  Discussion  of  Limitations 

Table  3  provides  a  summary  of  the  previous  studies  reviewed. 

This  subsection  discusses  the  limitations  of  these  studies  with  respect 
to  table  designs,  hypotheses,  statistics  investigated,  and  assessment 
criteria. 

Many  of  these  studies,  eight  out  of  15,  restricted  their  evalua- 
2 

tion  to  the  Pearson  X  statistic. .  Six  others  included  the  log-likeli- 
2 

hood  ratio  G  .  One  used  fullback's  minimum  discrimination  Information 

statistic,  21,  but  misapplied  it  to  the  homogeneity  test.  Some  of  the 

studies  included  other  statistics,  but  the  use  of  these  statistics  often 

was  restricted  to  small  tables  and/or  certain  types  of  hypotheses,  such 

2 

as  Goodman  and  Kruskal's  L  and  Freeman  and  Tukey's  T  .  tlone  of  the 
studies  included  the  GSK  procedure  and  statistic  in  their  evaluation. 

The  studies  were  very  selective  with  respect  to  the  hypotheses 


they  considered.  Some  considered  only  the  homogeneity  hypothesis. 

Others  considered  the  independence  hypothesis  but  required  all  the 
marginals  to  be  fixed,  which  severely  limited  the  application  of  their 
conclusions.  The  studies  that  did  consider  the  general  independence 
hypothesis  under  multinomial  sampling  were  usually  very  restrictive 
in  some  other  aspect.  Only  three  studies  included  the  no  second-order 
interaction  hypothesis,  and  none  evaluated  independence  hypotheses 
for  more  than  two  variables. 

While  most  of  the  studies  used  fairly  reasonable  criteria  for 
assessment,  some  of  the  studies  based  their  comparisons  on  Fisher's 
exact  test  or  similar  exact  tests.  Fisher's  exact  test  is  known  to 
be  conservative  and  is  based  on  fixed  marginals.  As  has  been  suggested 
[e.g.,  Berkson  (1978)],  it  may  not  provide  a  reasonable  criterion  for 
comparison. 

The  studies  were  all  somewhat  restrictive  in  the  size  of  the 
tables  (number  of  cells),  number  of  dimensions,  and  sample  size  consi¬ 
dered.  This  is  to  be  expected  because  of  the  nature  of  the  problem  and 
computer  time  limitations.  Matty  of  the  early  tests  considered  only  one 
or  two  table  sizes  and  few  sample  sizes.  One  of  the  larger  studies 
(with  respect  to  number  of  different  tables),  Roscoe  and  Byars  (1971), 
only  evaluated  the  homogeneity  hypothesis.  Only  three  of  the  studies 
included  three-way  tables.  Few  of  the  studies  considered  more  than 
four  or  five  different  sample  sizes. 

The  most  significant  limitation  of  these  studies  concerned  the 
underlying  probability  structures.  A  few  of  the  studies  were  restricted 
to  the  equlprobable  model.  Most  of  the  others  included  only  two  or 


three  different  probability  structures.  The  homogeneity  tests  gener¬ 
ally  had  tables  which  restricted  the  fixed  marginals  to  be  equal  or 
symmetric.  Some  of  the  studies  used  single  examples  with  only  one 
fixed  probability  structure. 

None  of  the  studies  combined  an  extensive  number  of  different 
table  sizes,  sample  sizes,  and  probability  vectors.  With  respect  to  the 

homogeneity  hypothesis,  Roscoe  and  Byars  (1971)  probably  did  the  best, 
combining  10  types  of  two-dimensional  tables,  six  sample  sizes,  and 
three  probability  structures.  Odoroff  (1970)  considered  a  reasonable 
range  of  table  designs  with  respect  to  his  MCE  calculations,  but  his 
study  was  limited  to  the  "no  second-order"  interaction  hypothesis.  None 
of  the  independence  studies  provided  a  reasonable  range  of  table  designs 

Especially  important  in  this  thesis  are  comparisons  across  tables 

under  the  independence  hypothesis.  With  the  exception  of  Craddock  and 

Flood  (1970),  no  study  provided  a  means  for  these  comparisons.  Craddock 

and  Flood's  study  was  limited  to  the  equiprobable  model.  Also,  their 

2 

conclusion  for  the  performance  of  the  Pearson  X  in  larger  tables  was 
based  on  a  very  subjective  analysis.  What  appears  to  be  needed  is  a 
more  objective  criterion  for  comparisons  across  tables  and  a  more 
extensive  consideration  of  table  sizes,  sample  sizes,  and,  especially, 
probability  vectors.  This  will  be  provided  by  the  research  of  this 


thesis 


Table  3.  Summary  of  Previous  Studies 
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CHAPTER  III 


CURRENT  METHODOLOGIES 


This  chapter  presents  and  discusses  the  most  popular  methodolo¬ 
gies  for  the  analysis  of  contingency  tables.  The  first  four  sections 
describe  and  relate  the  underlying  sampling  models,  the  log-linear 
models,  the  hypothesis  tests,  and  the  maximum  likelihood  estimates. 
Particular  emphasis  is  given  to  two-  and  three-way  tables  and  hypo¬ 
theses  of  independence,  homogeneity,  no  second-order  interaction,  and 

complete  independence.  The  last  three  sections  present  the  three  com- 

2 

petlng  statistics:  the  Pearson  X  ,  the  Kullback  minimum  discrimination 
information  statistic,  and  the  Grizzle,  Starmer,  and  Kock  (GSK) 
statistic.  The  theoretical  developments  of  the  Kullback  and  GSK 
statistics  are  presented  in  detail. 


3.1  Sampling  Models 

Most  contingency  tables  are  a  result  of  sampling  from  a  larger 
-■ulation.  The  type  of  sampling  model  will  determine  a  sampling  dis¬ 
tribution  for  the  observations.  The  four  most  common  sampling  distri¬ 
butions  are  the  Poisson,  the  multinomial,  the  product-multinomial,  and 
the  hypergeometrlc.  These  sampling  models  and  corresponding  distributions 
relate  directly  to  the  restrictions  placed  on  the  total  sample  size  and 
marginal  sums. 

If  there  are  no  restrictions  placed  on  the  total  sample  size  or 
marginal  sums,  and  observations  are  made  over  a  fixed  period  of  time. 


each  cell  has  an  Independent  Poisson  distribution.  The  density  func¬ 
tion  for  each  cell  of  the  two-way  r  x s  table  is 


r  s  x  -m 

f(x  )  =  n  n  (m  e  1J)/x  !  ; 

2  i-1  j-1  12  2 


i=l,2,...,r;  j-1,2, . . . ,s . 


(3-1) 


If  only  the  total  sample  size  (N)  is  fixed,  the  distribution  of 
the  cells  is  multinomial,  and  the  density  function  for  each  cell  of  an 
r  x  s  table  is 
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(3-2) 


If  each  category  of  one  of  the  variables  has  a  fixed  sample  size 
(marginal  sum),  then  the  sampling  distribution  is  product-multinomial, 
and  the  density  function  for  each  cell  of  an  rxs  table  is 
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j-1,2, . . . ,s.  (3-3) 


If  each  category  of  all  the  variables  have  a  fixed  sample  size 
(i.e.,  all  marginal  sums  are  fixed),  then  the  sampling  distribution  is 
hypergeometric,  and  the  density  function  for  each  cell  of  an  r * s  table 
is 

r  s  .  r  s 

f(x,Jx,  ,x  .)  ■  II  x.  !  II  x  J/N!  IT  II  x.  .!  ; 
lj  i-  -J  i-i  1-  j-i  i-i  j-i  « 

i=l,2,...,r;  j=l,2,...,s.  (3-4) 


As  might  be  expected,  these  distributions  are  highly  interrelated. 

If  X^  are  independent  Poisson  random  variables  with  density  function 

given  by  (3-1),  then  the  condtional  distributions  of  the  X  given 
r  s 

N  -  \  \  X. .  are  the  multinomial  distribution  of  (3-2).  When  sampl- 

i-1  j-1 

ing  under  the  product-multinomial  distributions  of  (3-3),  the  distribu¬ 


tions  of  the  marginals  (x^)  are  multinomial  with  density  functions 
similar  to  (3-2), 


f(x  .)  ■  N!  N  N  II  m.^/x  .!  ; 

•j  j-i  «  -J 

j»l,2,...,s.  (3-5) 

When  sampling  under  the  product-multinomial  distribution  with  x^  fixed, 
and  then  fixing  the  other  marginals  (x^),  the  corresponding  conditional 
distributions  are  hypergeometric  as  given  in  (3-4). 


The  multinomial  sampling  model  will  be  used  for  the  study  in 


this  thesis.  As  will  be  discussed  in  Section  3,  this  model  is  appro¬ 
priate  for  the  independence  hypotheses  of  this  study,  while  the 
product-multinomial  model  corresponds  to  homogeneity  hypotheses. 

Little  is  to  be  gained  by  separately  considering  Poisson  or  hypergeo¬ 
metric  sampling.  Under  Poisson  sampling,  the  hypotheses,  the  maximum 
likelihood  estimates,  and  the  chi-square  statistics  correspond  exactly 
to  either  the  product-multinomial  or  multinomial  cases.  As  previously 
noted  in  Chapter  II,  the  hypergeometric  sampling  plan  is  rarely  used 
in  practice.  Under  those  rare  situations  where  both  marginals  of  a 
two-way  table  are  fixed,  Fisher’s  "exact"  test,  based  on  the  hypergeo¬ 
metric  probabilities,  is  appropriate  for  the  independence  hypothesis. 

3.2  Log-Linear  Model 

As  previously  mentioned  in  Chapter  II,  the  log-linear  model  forms 
the  basis  for  much  of  the  work  in  contingency  table  analysis.  Maximum 
likelihood  estimates  and  minimum  discrimination  information  estimates, 
under  the  sampling  models  discussed  and  for  appropriate  hypotheses, 
have  been  fully  developed  for  this  model  and  are  relatively  easy  to 
obtain;  whereas,  other  models  have  suffered  from  the  complexity  of 
solving  for  appropriate  estimates. 

The  GSK  procedure  does  allow  for  both  the  linear  and  log-linear 
models  to  be  handled  in  a  similar  manner.  G.  C.  Koch  and  others  have 
extensively  used  the  linear  (additive)  model  [e.g.,  Johnson  and  Koch 
(1970),  Koch  and  Reinfurt  (1971),  and  Koch,  Johnson,  and  Tolley  (1972)]. 
The  advantage  of  this  model  seems  to  be  associated  with  Incomplete 


tables,  where  structured  zero  cells  are  present  [e.g.,  see  Koch  and 
Reinfurt  (1971),  Bishop,  Fienberg,  and  Holland  (1975,  Chapter  5),  and 
Chen  and  Fienberg  (1976)].  In  a  recent  book  Forthofer  and  Lehnen 
(1981)  state  that  the  log-linear  (multiplicative)  model  is  preferred 
when  events  are  "extreme";  i.e.,  when  some  cell  probabilities  are 
relatively  small  or  large  (less  than  0.2  or  greater  than  0.8),  When 
dealing  with  large  contingency  tables,  frequently,  all  the  cell  pro¬ 
babilities  are  small,  especially  under  multinomial  sampling. 

The  log-linear  model  with  its  associated  multiplicative  inter¬ 
actions  will  be  used  for  this  study.  The  model  is  directly  analogous 
to  the  linear  model  for  analysis  of  variance  (ANOVA) .  For  a  two-way 
r  x  s  contingency  table  the  complete  or  fully  saturated  log-linear 
model  is 

in  Py  -  u  +  ul(1)  +  u2(j)  +  »12(ij); 

i“l»2, . • . ,r;  j*l,2,.. . ,s;  (3—6) 

where  u  =  the  grand  mean  of  the  logarithms  of  the  probabilities;  i.e. 

u  *  (l/rs)(£n  p^  +  £n  p12  +  •••  +  Prg)  •  (3-7) 

Also,  u  +  =  the  mean  of  the  logarithms  of  the  probabilities  at 

level  i  for  the  first  variable;  i.e., 
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u  +  ”  (1/s) (in  p^  +  in  Pi2  +  ...  +  in  Pig); 


i-1,2 . r.  (3-8) 


Similarly,  u  +  u2(j)  -  t^e  ®ean  of  the  logarithms  of  the  probabilities 
at  level  j  for  the  second  variable;  i.e.. 


u  +  -  (1/r)  (in  p^  +  in  p2j  +  ...  +  in  p^); 


j-1,2 . s.  (3-9) 


As  defined,  u^^  and  u2(j)  represent  deviations  from  the  grand 
mean.  Summing  (3-8)  over  i, 
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Similarly,  summing  (3-9)  over  j,  \  u«,..  ■  0.  These  results  are  anal- 

J-l 

ogous  to  the  familiar  constraints  for  the  ANOVA  linear  statistical 


model.  In  a  similar 


manner  the  constraints,  £  ui2(ij)  "  E  u12(ij)  “  0> 


are  placed  on  the  u^2(ij)’  anc*  t‘iese  terms  represent  the  Interaction 
between  the  variables,  analogous  to  interaction  in  ANOVA.  Continuing 
the  ANOVA  analogy  and  solving  for  the  terms  of  the  model,  from  (3-7), 
(3-8),  and  (3-9)  the  main  effects  are 
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Using  (3-6),  (3-7),  (3-10),  and  (3-11), 
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Degrees  of  freedom  for  these  terms  are  calculated  exactly  as 
In  ANOVA.  There  are  a  total  of  r.s  degrees  of  freedom,  one  associ¬ 
ated  with  each  observation  or  cell.  Each  u  term  has  one  degree  of 
freedom  associated  with  each  independent  parameter.  The  constraints 
reduce  the  number  of  independent  parameters.  Table  4  lists  the  degrees 
of  freedom  for  the  r*s  table. 

Table  4.  Degrees  of  Freedom:  r * s  Table 
u-term  degrees  of  freedom 
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The  extension  to  higher-way  tables  follows  as  easily  as  the 
extension  of  the  linear  model  for  ANOVA.  The  only  problems  involved 
are  the  interpretations  of  the  higher-way  interaction  terms.  An 
excellent  discussion  is  presented  in  Bishop,  Fienberg,  and  Holland 
(1975,  Section  2,4). 

Frequently,  it  is  more  convenient  to  work  in  the  frequency 
spectrum  (x^  and  m^).  When  this  is  the  case,  the  log-linear  model 
can  easily  be  modified.  For  multinomial  sampling 


Taking  logarithms  of  both  sides, 


in  ■  in  N  +  in  , 


The  log-linear  model  then  becomes 


in  »y  -  U  ♦  u1(1)  +  n2(j)  *  u12(1J)  +  2n  N 


+  Ul(l)  +  U2Cj)  +  U12(lj)> 


where 


u*  -  u  +  in  N. 


For  product-multinomial  sampling 


■ij  " 


in  m^  ■  in  +  in  p^ 


u  +  ui(i)  +  u2(j)  +  ui2(ij)» 


where 


*  u  +  in  m. 


The  ultimate  goal  In  the  analysis  of  contingency  tables  is  to 


determine  the  relationships  among  the  variables.  The  approaches  taken 
are  problem  dependent  in  that  they  depend  on  what  particular  relation¬ 
ship  is  of  concern.  Sometimes  a  simple  procedure  of  "smoothing"  the 
cells  of  a  contingency  table  to  correspond  to  some  fixed  constraints 
or  to  some  other  set  of  data  is  enough  to  indicate  the  desired  rela¬ 
tionships.  Other  times  certain  estimates  of  the  cells  are  calculated 
to  show  patterns  or  trends.  More  often  than  not  a  hypothesis  testing 
procedure  is  necessary  to  explore  variable  relationships.  The  types 
of  hypotheses  are  numerous,  and  the  literature  is  filled  with  extensive 
discussions  and  interpretations.  This  study  will  be  limited  to  examin¬ 
ing  the  small  sample  properties  of  current  methodologies  as  they  relate 
to  hypotheses  of  independence,  or  "no- interaction",  under  multinomial 
sampling.  These  hypotheses  will  be  discussed  in  this  section  together 
with  the  contrasting  homogeneity  hypotheses  under  product-multinomial 
sampling. 

All  these  hypotheses  correspond  to  setting  certain  terms  of  the 
log-linear  model  equal  to  zero.  To  demonstrate  these  relationships, 
consider  the  2*2  table  of  probabilities  in  Figure  3.  Under  multi- 
nomial  sampling  £  £  p.  ■  1,  and  the  hypothesis  of  concern  is  indepen- 

i  j  J 

dence,  or  no  interaction, between  the  variables  A  and  B.  The  marginal 
probabilities  P^.»  P2.»  P,^>  an<*  P.2  are  unconstrained  probabili¬ 
ties  of  being  in  a  certain  category  of  a  variable  (e.g.,  P(A-l)  *  P1#)» 
The  classic  sense  of  independence  would  require  that  P(AB)  *  P(A)P(B). 
The  corresponding  hypothesis  statement  for  the  2x2  contingency 


pll 

P12 

P21 

P22 

Figure  3.  2x2  Table  of  Probabilities 


table  is 


V  Pli  ■  »i.*.ji  1-1 • 2  !  l'1-2- 


For  the  2x2  table  it  is  easily  seen  that  the  hypothesis 
V  Pjj  *  pl.p.l»  is  aufficient  for  (3-14): 


P11  "  pl*p.l 


pl.(1“p.2)  *  pi.  “  pl.p.2* 


but  pu  -  px<  -  p12;  therefore. 


p12  -  pl-p«2  * 
pll  “  pl-p.l 

-  (l-p2<)p<:L  -  p#1  -  P2.P.1  , 
but  pxl  -  p<x  -  p21;  therefore, 

P21  "  p2,p,l  * 


(1-P2.) (1-P,2^ 


P11  "  pl-p.l 

*  1  -  P2.  “  P.2  +  p2-p.2» 

but 

P11  "  1  "  P12  “  P21  “  p22 

■  1  -  (P.2“P22^  "  ^P2.”P22^  "  p22 

"  1  ‘  P2.  “  p.2  +  p22  5 

therefore, 

P22  "  p2-p.2  * 

From  (3-6)  the  log- linear  model  for  the  2  *  2  table  is 


in  Plj  -  u  +  u1H)  +  «j(j)  +  ul2(Ij) 


;  1*1,2;  j— 1, 


with  constraints, 

ul(l)  +  ul(2)  "  0 

U2(l)  +  u2 (2)  *° 

U12(ll)  +  u12(21)  "  ° 

u12(12)  +  u12(22)  "  0 

U12(ll)  +  u12(12)  "  0 

u12(21)  +  u12 ( 22)  "  °* 


The  following  relations  exist: 


Ul(l)  *"U1(2)’  u2(l)  *  ‘U2(l)’ 

and 

U12(ll)  "  “u12(12)  =  "u12(21)  *  u12(22)  * 
Reparameterizing  by  letting 

Ul(l)  “  V  u2(l)  *  u2 *  and  U12(ll)  “  u12  » 
the  logarithms  of  the  four  cell  probabilities  are 

In  p^  »  u  +  +  u2  +  u12 

?12  '  u  +  U1  -  “2  -  u12 

in  P2i  *  u  “  ui  +  u2  “  ui2 

in  p22  -  u  -  -  u2  +  u12  . 

Solutions  for  the  "u"  parameters  can  be  obtained  from  (3-15). 
the  four  equations, 

4u  -  2,n  p^  +  in  p^  +  P21  +  P22  * 

so  that, 

u  -  j  (Jin  pn  +  £n  p12  +  in  p21  +  in  p22)  . 


(3-15] 


Summing 


(3-16 
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J 


J 

1 

i 


Substituting  (3-16)  into  the  first  and  second,  and  summing, 

2ul  +  (£n  P11  +  £n  P12  +  £n  P21  +  ln  P22^  *  ln  pll  +  Zn  p12  5 

therefore, 

u1  *  ■£  (Jin  pu  +  Jin  p12  in  pn  -  in  p^)  •  (3-17) 

Substituting  (3-16)  into  the  first  and  third,  and  summing, 

2u2  +  2[-|(in  pu  +  Jin  p12  +  Jin  p21  +  Jin  p22)]  -  Jin  pn  +  Jin  p21  ; 
therefore, 

u2  -  j  (Jin  pu  -  in  p12  +  Jin  p21  -  Jin  p22>  •  (3-18) 

Summing  the  first  and  fourth  equations,  and  subtracting  the  second  and 
third  equations, 

4ui2  ■  Jin  p^  -  in  p12  -  in  p21  +  in  p22  ; 

so  that, 

u12  "  i(iln  P11  "  *n  pi2  ■  P21  +  ln  p22)  • 


(3-19) 
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Under  the  hypothesis  of  no-interaction,  (3-14), 

u12  -  j  (to  p1#P#1  -  Jin  p1#p>2  -  An  p^p^  +  An  P2.P.2> 

*  J  <to  P1#  +  to  P.x  “  to  P1#  -  to  p2  -  An  p2<  -  An  P<1 
+  An  p2<  +  An  p>2) 

-  0. 

Conversely,  if  u^2  ■  0  then 

to  Pu  “  to  p12  -  An  p21  +  An  p22  -  0 

and 


which  implies  that 


P11P22  ,  1 
P12P21 


or 


Making  substitutions, 


pll(1"pll"p12_p21)  '  <pl-"pH)(p-l_pll> 

pu(1-pii'<pi.’pu)  ‘  <p.rpu))  ■  pi.p.i  -  pn<pi.+p.rpii) 
pii(1-pi.-p.i+pu)  ■  pi.p.i  -  pu(pi.4p.rpu) 
pu  ■  pl.p.l> 

and  independence,  (3-14),  follows.  Thus, 

HQ:  u12  =  0  (3-20) 

is  equivalent  to  (3-14)  for  the  2X2  case.  In  general,  for  the  rxs 
case 


V  U12(ij)  =  0;  (3-21) 

Consider  now  the  2x2  table  under  product-multinomial  (product- 
binomial)  sampling,  where  the  marginal  observations  for  one  of  the 
variables  are  fixed,  say  x^  and  x2<  .  Thus,  the  corresponding  pro¬ 
babilities,  and  p2>,  are  equal  to  one,  and  the  2X2  table  is  given 


in  Figure  4. 


6 


P21 

Figure  4.  2x2  Table:  Product-Binomial  Sampling 

The  hypothesis  of  concern  is  homogeneity;  that  is, 

V  P11  =  P21  =  P  0r  p12  “  P22  =  1  “  P>  (3-22 

where  p  is  usually  not  specified.  The  complete  log-linear  model, 

(3-6),  under  product-multinomial  sampling  is 

An  Pij  *  U  +  Ul(i)  +  U2(j)  +  U12(ij);  i*1,2;  j=1’2» 

where  now  p^2  *  1  -  and  p22  "  1  -  P21*  Using  the  parameterization 
in  (3-15)  and  the  relations  in  (3-7),  (3-8),  and  (3-9), 

u  -  j  (Zn  pn  +  Zn  P21  +  in(l-p21)) 

-  i  (Zn  Pn(l-Pn)  +  ^  P21(1"P21));  0-23) 

U  +  U1  "  I  (in  P11  +  Zn  P12)  “  J  in  P11(1"PH); 


so  that. 


U1  ■  4  (5-n  P11(1“P11>  "  p21(1_p21));  (3-24) 

u  +  u2  "  I  pH+in  p21)  ”  \  ln  pllp21’ 

so  chat, 


u2  "  4  (*n  pnp2i  **  ^(l-p1]L)(l-p21)) 


i  taL-tfft - vl  ; 

4  l(1"Pll)(1"P21) J 


(3-25) 


and 


u12  *  *  Un  PU  ' 


Jin  p21  +  £n(l-p21)) 


p2i<i-p 


21 

11 


(3-26) 


The  log-linear  model  is  not  the  most  convenient  model  for  des 


cribing  this  product-binomial  situation.  The  model  usually  used  is 
the  logit  model,  defining  the  relative  proportions  in  the  rows, 


+  v*2  ■  0. 

Under  the  homogeneity  hypothesis,  (3-22),  the  2x2  table  is  given 
in  Figure  5. 

1 
1 

Figure  5.  2x2  Table:  Homogeneity  Hypothesis 

For  the  log-linear  model  from  (3-23)  through  (3-26), 


U  -  jr  (In  p(l-p)  +  Jin  p(l-p))  =  Y  £n  p(l-p) 

Uj  »  ^  (Jin  p(l-p)  -  Jin  p(l-p))  *  0 
2 

“2 "  *  Ka^)1] "  ^  ln(p/a"p>> 

ui2’£*'n[$i$]-i2n1-0- 

For  the  logit  model  from  (3-27) , 


w  =  2u„  =  Jin  ~— 
L  1-p 


*1  ‘  2u12  '  0 


W2  '  *2u12  ‘  0 


■  W. 


Therefore,  the  homogeneity  hypothesis  corresponds  to  H^:  *  0 

in  the  log- linear  model  and  w^  ■  *  0  in  the  logit  model.  The 

convenience  of  the  logit  model  is  obvious.  In  general,  the  logit  model 
is  preferred  in  most  "stratified"  sampling  situations  where  all  but 
one  set  of  marginals  are  fixed. 

These  hypotheses  can  be  extended  to  higher-way  tables,  although 
it  wasn’t  until  Bartlett  (1935)  that  a  precise  definition  of  "no  higher 
way"  interaction  in  a  three-way  table  was  given.  In  the  three-way 
table  the  hypothesis  of  "no  second-order"  interaction  as  defined  by 
Bartlett  (1935)  for  a  2^2><2  table  is 


H0:  P111P122P212P221  "  p112p121p211p222* 


(3-28) 


and  expanded  to  a  general  rxsxt  table  by  Roy  and  Kastenbaum  (1956)  is 

H0!  prstpijtprskprjk  *  pi3cprjtprskpijk;  (>29) 

i*l,2,...,(r-l) ;  j-1,2, . . . , (s-1) ;  k-1,2, . . . , (t-1) . 

Considering  the  2x2x2  case,  the  log-linear  model  can  be  para¬ 
meterized  as  in  (3-15)  with 

U1  *  Ul(l) 

U2  *  U2(l) 
u3  “  U3(l) 

“12  *  “12(11)  (3-30) 

u13  *  u13(ll) 

U23  *  u23(ll) 
u123  *  u123(lll)  5 

so  that,  for  example. 


•  ill 


Using  matrix  notation, 


&n  p  »  X  B> 


(3-31) 


where  in  lexicographic  order 


(£n  £)’  -  (£n  pm,2n  P112*««*»an  P222^  ’ 

3  *  * ^2 *^3 * ^12 * ^13  * ^23  * ^123^ * 

and 

1  1  1  1  1  11  1 

1  1  1-1  1  -1  -1  -1 

1  1-1  1-1  1-1-1 

1  1  -1  -1  -1  -1  1  1 

1-1  1  1-1-1  1-1 

1-1  1-1-1  1-1  1 

1-1-1  1  1-1-1  1 

1  -1  -1  -1  1  1  1  -1  . 

Note  that  by  factoring  /8  out  of  X  the  result  is  an  orthogonal  matrix 


x  *  /8  X]_  ; 

so  that, 

2n  £  -  /8  XL  &»  (3-32 


and  solving  for 


Therefore,  in  particular. 


u123  “  8  (2n  plll  “  p112  "  2n  p121  +  p122  "  2,11  p211 

+  In  p212  +  J,n  p221  -  In  p222> 


1  „  /  P111P122P212P221 
x,n 


8  \P112P121P2UP22 


■3- 


(3-34) 


From  (3-28)  and  (3-34)  the  no  second-order  interaction  hypothesis  is 
clearly  equivalent  to 


H 


0 


y123 


-  0. 


Product-multinomial  sampling  in  three-way  and  higher  tables  allows 
for  several  homogeneity-type  hypotheses  depending  on  whether  one-way, 
two-way,  or  higher  marginals  are  fixed,  and  how  many  of  these  marginals 
are  fixed.  One  of  the  most  common  situations  involves  a  series  of 
two-way  tables  with  fixed  sample  size. 

Consider  a  2x2x2  contingency  table  with  one-way  marginals 
fixed.  One  appropriate  hypothesis  is  to  test  for  homogeneity  of  the 
two  2x2  tables  formed  for  each  i. 


p2jki  J-l,2,i  k-1,2. 


(3-35) 


These  2x2  tables  are  given  in  Figure  6 


rZSSv’i 


k 

1  2 
plll  P112 

p121  P122 
i-1 


k 

1  2 


P211 

P212 

P221 

P222 

Figure  6.  2*2x2  Table:  Fixed  One-Way  Marginals  (xi#<  ) 

Under  this  hypothesis  the  solutions,  (3-32),  for  the  u^3»  u^^»  u^2* 


u^  terms  of  the  log-linear  model  are 


1  /PlllP122P212P22l\  b  Q 

8  \P112P121P211P222/ 

1  ^YP111P121P212P222\  ,  0 
8  \P112P122P211P221/ 


I  JUYP111P112P221P222 

8  v  121P122P211P212 


1  .  /P111P112P121P122 
8  \P211P212P221P222 


^  *  0. 


Conversely,  given  u^3  ■  *  ui  “ 

P111P122P212P221  "  P112P121P211P222 
P111P121P212P222  *  P112P122P211P221 
plllp112p221p222  "  P121P122P211P212 


(3-36) 


(3-37) 


(3-38) 


ji 
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P11XP112P121P122  *  P211P212P221P222‘  (3-39) 

Substituting  for  p222  in  (3-37)  from  (3-36), 

p,,  p  D  / P111P122P212P221\ 

111  121  212  ^  P112P121P211  j  ~  P112P122P211P221; 

so  that. 


P111P222  *  P122P211  *  (3-41) 

Substituting  for  p222  in  (3-36)  from  (3-38), 


9 


v111f221 


h121h211 


V  J 


Combining  (3-40),  (3-41),  and  (3-42), 

plll(p212+p221+p222)  =  p211(p112+p121+p122) *  (3-43) 

Since  the  marginals  and  *2-.  are  ^xe<^»  the  corresponding  marginal 

probabilities,  P1>#»  p2..,  equal  one*  Substituting  into  (3-43), 

plU<1-p211>  *  p211(l-',lU) 

P1U  *  P1UP211  ‘  p211  "  p211plll’ 


so  that. 


plll  “  p211* 


Similar  calculations  are  easily  performed  to  verify  that  p 
p2jk  ^0r  ^  an<*  Therefore, 


Ijk 


H0:  u123  “  u12  "  u13  *  U1  "  0 


(3-44) 


is  equivalent  to  (3-35). 

The  two  tests  described  above  (second-order  interaction  and  homo¬ 
geneity)  are  the  highest  order  tests  possible  under  the  given  sampling 
constraints.  In  general,  a  hierarchical  set  of  hypothesis  tests 
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similar  to  procedures  for  stepwise  regression  can  be  performed  on 
any  collection  of  data.  Birch  (1963)  in  his  presentation  of  the  log- 
linear  model  was  one  of  the  first  to  describe  this  hierarchical 
structure.  Goodman  (1971a)  presented  an  excellent  discussion  of  this 
structure,  along  with  the  appropriate  hypothesis  tests,  relating  the 
procedures  to  several  techniques  of  stepwise  regression.  Each  of 
these  tests  corresponds  to  the  absence  of  certain  "u  terms"  of  the 
log-linear  model.  With  this  log-linear  model  the  interpretation  of 
these  tests  become  relatively  simple. 

Besides  "no  second-order"  interaction  under  multinomial  sampling 
for  three-way  and  higher  tables,  other  independence- type  hypotheses  in¬ 
clude  pairwise,  conditional,  partial,  mutual,  multiple  and  complete 
independence.  Of  particular  Interest  in  this  thesis  is  complete  inde¬ 
pendence. 

Complete  independence  specifies  that  all  variables  are  indepen¬ 
dent  at  all  levels.  It  should  seem  obvious  that  this  would  require 
all  interaction  terms  of  the  log-linear  model  to  be  equal  to  zero.  To 
demonstrate  this,  consider  the  classic  sense  of  complete  Independence 
of  three  variables  A,  B,  and  C, 

P(AflBnc)  -  P(A)P(B)P(C) . 

For  a  three-way  contingency  table  this  corresponds  to 


Pi..P. 


i«l,2,...,r;  J-1,2,, 


(3-45) 
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In  particular,  consider  the  2x2x2  table.  Using  the  matrix  solution, 
(3-33),  of  the  log-linear  model  and  the  null  complete  independence 
hypothesis,  (3-45), 


1  „  /P111P122P212P221 
u„.  ■  q-  x-n  - 

123  8  \P112P121P211P222 


JL  £n  /Pl« .p»l.p. »lpi» .p«2«p«  «2P2« •P»1»P«»2P2«  »P»2«P«  »1 
8  lpi..P.i.P.  2P1**P*2.P-.1P2..P.1.P.-1P2..P*2.P..2 


Similarly,  it  can  be  shown  that  u^  *  u^  *  U2^  “  0*  Conversely, 
given  ^23  *  *  u13  *  u23  *  the  relationshiP  in  (3-45)  can  be 


verified  for  all  i,  j,  k.  Thus,  Hn:  u 


123  "  U12  =  u13  “  u23 


equivalent  to  the  complete  independence  hypothesis  (3-45). 


3.4  Maximum  Likelihood  Estimates 
Maximum  likelihood  estimates  (MLE)  as  discussed  in  Chapter  II 
are  theoretically  satisfactory  for  analyzing  contingency  tables,  al¬ 
though  other  best  asymptotically  normal  (BAN)  estimates,  as  described 
by  Neyman  (1949),  have  similar  properties.  For  contingency  tables  MLE 
are  of  two  types:  those  that  can  be  computed  directly  and  those  that 
require  some  iterative  scheme.  Whether  MLE  can  be  computed  directly 
or  must  be  computed  iteratively  depends  upon  the  null  hypothesis  to  be 
tested  and  the  size  of  the  contingency  table. 

Considering  the  multinomial  sampling  distribution,  the  maximum 
likelihood  estimates  for  the  two-way  r»s  table  under  the  hypothesis 


of  independence  can  easily  be  derived.  Following  the  usual  procedure] 
the  multinomial  density  for  each  cell  will  be  used  instead  of  the 
joint  likelihood  density.  The  kernels  are  equivalent,  and  both  lead 
to  the  unique  MLE. 

The  multinomial  density  for  each  cell  of  a  two-way  r  x  s  table 
was  given  in  (3-2), 


f(x±j)  =  N!  N 


-r 


r 

n 


II  m. 


ij 


i-1  J-l 


ij 


i-1, 2,. 


, r ;  j =1 ,2, . . . ,s; 


with  multinomial  sampling  constraint. 


l  l 


i-1  j-l 


r  s 

l 

i-1  j-l 


l  * 


=  N. 


Taking  the  logarithm  and  using  a  Lagrangian  multiplier  (1)  to  account 
for  the  constraint,  the  log-likelihood  function  is 


L  -  2n(N!)  -  R  in  N  -  An(x  !)  +  I  £  An  tn  +  X(J  m  -N)  .  (3- 

ij  ±  j  ij  i  j 


Under  the  hypothesis  of  independence 
log-linear  model  is 


U12(ij)  * 


0,  and  from  (3-13)  the 


or  equivalently 


"ij  - 

=  exp  u  exp  u^^exp  u2^. 


Substituting  into  (3-46), 


L  =  Jin  N!  -  N  Jin  N  -  Jln(xij!)  +  £  £  x^  (u+u^^+u^ ^ ) 


+  ^ I  I  (exp  u  exp  u1(i)e*P  u2(j)  ~  N) • 


(3-4; 


The  maximum  likelihood  estimates,  m^  »  expCu+u^^+u^^) ,  can  now 
be  derived  by  taking  partial  derivative  of  (3-47)  with  respect  to  the 
variables  u,  u^^,  u2(j)’  and  *  and  settin8  them  equal  to  zero: 


|jf  -  N  +  X  exp  u  l  exp  u1(i)  £  exp  u2(J)  -  0 


(3-4i 


3L 


3u 


Ki) 


ci.  +  X  exp  G  exp  Ql(i)  l  exp  Q2(j)  "  0; 


j 


i*l,2, . . . ,r 


(3-4' 


3L 


3u 


2(j) 


x.j  +  X  exp  u  exp  u2^  l  exp  u^j  -  0; 


j“l»2, ... ,2 


It  -  exp  u  y  exp  G,  y  exp  G,,4.  -  N  -  0. 


(3-5' 

(3-5 


From  (3-48)  and  (3-51) 


N  +  XN  -  0, 


which  implies  that 


X  *  -1. 


Substituting  into  (3-49)  and  (3-50), 


i#  -  exp  u  exp  u1(i)  £  exp  u2(i);  i-1, 


2(j) 


.j  "  exP  «  exp  l  exp  u1(i);  j-1. 


Then 


x.  x  . 

i#  *J. 


N 


exp  u  exp  1  exP  u^^exp  ^  exP  u 


exp  u  l  exp  u^^  V  exp  u2 


exp  u  exp  u1^i^ exp  U2(J)‘ 


Therefore,  the  MLE  are 
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Under  the  independence  hypothesis  these  maximum  likelihood 
estimates  are  unique  for  the  three  sampling  models,  Poisson,  multi¬ 
nomial,  and  product-multinomial  [e.g.,  see  Birch  (1963),  Bishop  (1969a), 
and  Haberman  (1974b) ] .  As  discussed  by  Goodman  (1971a)  and  shown  by 
Haberman  (1974a),  the  MLE  are  unique  under  any  of  the  Birch  hierarchical 
hypotheses. 

As  has  been  shown,  the  two-way  table  permits  a  direct  closed- 
form  formula  for  the  maximum  likelihood  estimates  under  the  independence 
hypothesis.  For  higher-way  tables  and  the  hypothesis  of  no  "higher- 
order"  interaction  this  is  not  the  case.  Consider  the  r*s*t  three- 
way  table  and  the  log-linear  model  under  multinomial  sampling, 

fn  mijk  -  u'  +  u1(i)  +  u2(j)  +  u3(k)  +  u12(ij)  +  u13(ik) 

+  U23(jk)  +  U123(ijk) 5 

i-l,2,...,r;  j-l,2,...,s;  k-l,2,...,t.  (3-53) 


The  hypothesis  of  no  second-order  interaction  sets  u^23(ijk)  to 

zero  for  all  i,  j,  and  k.  Under  this  hypothesis  there  is  no  direct 
formula  to  calculate  the  MLE.  There  are  two  iterative  procedures  cur¬ 
rently  used.  Iterative  Proportional  Fitting  (IPF)  and  a  Newton-Raphson 
(N-R)  procedure.  Both  procedures  converge  to  the  unique  MLE  for  hier¬ 
archical  hypotheses.  Haberman  (1974)  extensively  discusses  convergence 
rates  and  the  other  advantages  and  disadvantages  of  each  procedure. 

This  study  will  use  the  IPF  procedure. 


For  the  two-way  table  under  the  hypothesis  of  independence, 
the  maximum  likelihood  estimates  were  derived  above  and  given  in 


(3-52).  Summing  (3-52)  over  i. 


x  x  ./N 

..  .j 


» 


and  summing  (3-52)  over  j, 


m.  -  x,  x  /N  •  x, 
i*  i«  ••  i» 

Note  that  the  maximum  likelihood  one-way  marginals  will  equal  the  ob¬ 
served  one-way  marginals.  If  this  was  known  in  advance,  the  MLE  could 
be  derived  from  these  "sufficient"  statistics  by  fitting  the  table  of 
MLE  to  the  observed  marginals  in  an  iterative  fashion.  Begin  by 
assuming  initial  values,  {m^ } ,  for  the  estimates.  Then,  use  the 
following  formula  to  fit  the  table  to  the  observed  marginals  x^: 


m 


a) 

ij 


Note  that 


*<1> 

mi. 


*(1) 

mij 


l  ® 


s(°)x 

ij  i 


/<0) 

i* 


*(0) 
m'  'x, 
!•  i  < 


Next,  fit  the  table  to  the  observed  marginals  x^  with  the  formula 


Note  that 


•  »  i  •  .  ».  *  . 


ij  U  *-j'  «j 


fi(2>  .  T  fi<2)  -  7  fid),  .  ^lK  /*<!>  .  x 

•j  £  ij  }  “ij  *«j'  *J  *j  \T  \J  *j* 


Now  calculate  with  initial  estimates,  m^  -  1,  for  all  i  and  j. 


1  *  x .  /s  -  x.  /a 
i»  i» 


J  x.  /s  *  x  /s  -  N/s 
£  !•  •  • 

(xi#/s)(x#j)/(N/S)  -  x±<x ^/N, 


which  is  the  exact  closed-foroed  formula,  (3-52),  previously  derived. 

For  the  three-way  r  * s  x  t  contingency  table,  formulas  for  the 
no  second-order  interaction  hypothesis  are: 


aU)  .  fi(0)  ,*«». 

ttijk  “ijk  (*iJ./mU*) 

a(2)  .  fiU>  /oCDs 

“ijk  “ijk  (xi-k/ai.k) 

o(3)  _  a(2)  f  /fi<2K 
“ijk  “ijk  jk  • jk' * 


(3-54) 


However,  as  previously  stated,  no  dosed-formed  solution  exists.  The 
three-step  cyclic  process  above  would  have  to  be  repeated  until  some 
desired  accuracy  is  obtained*  As  many  authors  have  shown  [e.g.,  Brown 


-jr~— *  - •  ••  »  r-»-  •  -  v -■ *  *•  *  »■' 


(1959),  Fienberg  (1972a, b),  and  Haberman  (1974a)]  these  estimates  will 
converge  to  the  unique  maximum  likelihood  estimates. 

For  the  three-way  r  x s  *  t  table  under  complete  independence, 
the  one-way  marginals  provide  the  sufficient  statistics  to  determine 
uniquely  the  MLE.  The  closed-form  formula  follows  the  form  of  the 
hypothesis  (3-45), 

®ijk  "  Xi»*X»j*X*«k’  J-l,2,...,sj  k«l,2,...,t.  (3-55) 

2 

3.5  Pearson  Chi-Square  (X  ) 

Over  the  years  the  Pearson  chi-square  statistic, 

X2-I  (Xj_  -  mi)2/mi,  (3-56) 

where  the  m^  are  the  expected  values  under  the  null  hypothesis,  and  the 
sum  is  taken  over  all  combinations  of  categories  (cells  for  a  contin¬ 
gency  table),  has  been  the  most  popular  statistic  for  testing  a  number 
of  different  types  of  hypotheses.  As  used  with  contingency  tables, 
the  practice  has  been  to  replace  the  m^  with  their  maximum  likelihood 

estimates  (MLE)  under  the  particular  hypothesis.  Another  procedure  is 
2 

to  minimize  X  with  respect  to  the  m^,  consistent  with  the  given  hypo¬ 
thesis.  Neyman  (1949)  showed  that  this  procedure  yielded  BAN  estimates 
with  properties  similar  to  the  MLE.  The  problem  with  the  procedure 
is  its  computational  complexity. 

For  example,  consider  the  two-way  2><2  table  under  multinomial 


sampling  and  Che  hypothesis  of  independence.  Reparameterizing  the 


{p^j}  table  to  account  for  the  sampling  procedure  and  the  independence 
hypothesis,  let  p  -  p^  and  q  *  p#^,  then  m  l-p  and  P.2  M  l-q* 
Under  the  Independence  hypothesis 

H0:  »ij  ■  ‘-1-2!  3-!.2i 

and  the  multinomial  table  in  Figure  7 


P2. 

1 

Figure  7.  2^2  Table:  Multinomial  Sampling 

becomes  the  reparameterized  table  in  Figure  8. 


P 

1-P 
1 

Figure  8.  2^2  Table:  Independence 

Hypothesis  Reparameterized 

Recalling  that  m^  ■  Np^  for  multinomial  sampling,  the  Pearson  statis¬ 
tic,  (3-56),  becomes 


pq 

p(i-q) 

q(l-p) 

(l-p) (l-q) 

q  l-q 


pll 

P12 

P21 

P22 

P*1  p.2 
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fcl 

M 


x  -n  /»pir 


(3-57) 


Substituting  the  reparameterized  p^  into  (3-57) , 


(Xn-Npq)2  (xl2-Np(l-q))2  (x2]-N(l-p)q) 2 

Npq  +  Np(l-q)  +  N(l-p)q 

(x22-N(l-p)(l-p))2 
+  N(l-p) (1-q)  * 


(3-58) 


To  solve  for  the  m^  which  minimize  X  ,  take  partial  derivatives  of 
(3-58)  with  respect  to  p  and  q  and  set  equal  to  zero.  Solve  these 
equations  for  the  estimates  (p,q)  of  (p,q).  These  can  then  be  convert¬ 
ed  with  the  reparameterized  2  x  2  table.  Figure  8,  to  estimates  p  of 

p  and  used  to  'find  m.  .  ■  Np. ..  Unfortunately,  the  solution  of  these 

13  lj 

equations  requires  solving  a  fifth-order  polynomial.  Larger  tables 

would  require  even  more  difficult  computations. 

The  simplicity  of  the  MLE  as  well  as  their  good  asymptotic 

properties  make  them  the  choice  for  most  situations.  In  this  study 

2 

the  MLE  will  be  used  with  the  Pearson  X  statistic. 


3.6  Minimum  Discrimination  Information  (MDI] 


3.6,1  General  Derivation 

In  general,  the  minimum  discrimination  information  (MDI)  approach 
to  the  analysis  of  contingency  tables  uses  a  measure  of  information 
developed  by  Kullback  (1959),  and  minimizes  this  measure  with  respect 
to  the  hypothesis  of  concern.  The  measure  is  then  used  as  a  statistic 


90 


to  test  the  null  hypothesis.  Using  notation  from  Gokhale  and  Kullback 
(1978) ,  let  p(w)  and  tt(w)  be  the  probabilities  associated  with  two 
contingency  tables  defined  over  the  set  of  cells  Q,  where  £  p(u)  * 

n 

£  tt(co)  -  1  (corresponding  to  the  multinomial  sampling  model).  The  dis- 
fi 

crimination  information  between  £  and  tt  is  defined  as 

I(£:tt)  -  l  p(w)  In  Cp(£u)/Tr(w)) ,  (3-59) 

Q 

where  tt  is  fixed  and  depends  on  the  particular  problem,  and  p  ranges 
over  a  family  of  distributions,  P,  which  relate  to  the  hypothesis  of 
concern.  Kullback  (1959)  proves  several  useful  properties  of  this 
measure.  In  particular, 

nonnegativity  -  I(£JJ[)^0  and  ■  0  if  and  only  if  £  •  tt;  (3-60a) 

convexity  -  I(£!£)  is  a  convex  function  of  £;  (3-60b) 

nonsymmetry  -  I(£;tt)  t  I(tt:£)  in  general;  (3-60c) 

and  that  I(£:J[)  is  a  measure  of  the  deviation  between  the  distributions. 

The  principle  of  MDI  estimation  is  to  minimize  (3-59)  over  the 
family  of  distributions,  P,  which  satisfy  certain  linearly  independent 
constraints  related  to  the  hypothesis  of  concern.  Continuing  with  the 
notation  of  Gokhale  and  Kullback  (1978a),  these  constraints  are 
expressed  in  matrix  form. 


C  £  -  6, 


(3-61) 


where  £  is  an  (n+1)  *  "design  matrix  ,  £  is  the  Q  x  l  probability 
matrix  (cell  probabilities) ,  and  £  is  an  (n+1)  *  1  matrix  of  con¬ 
straining  values.  The  rank  of  £  is  n+1  (i.e.,  the  rows  are  linearly 
Independent).  The  solution  can  be  obtained  by  use  of  Lagrangian 
multipliers.  The  nonlinear  optimization  problem  is 

minimize  f(j>)  *  £  p(o>)‘  Jin  (p(w)/7r(w)), 

Q 

subject  to  £  £  ■  £ 

£  >  0  .  (3- 

The  constraints  can  be  written  as 


I  (^(uOpfa)  -  9k;  k»0,l,...,n.  (3- 


Using  Lagrangian  multipliers,  A.  ,  (3-62)  becomes 


minimize  F(£)  »  £  p(w)  Jin  (p(w)/iT(w)) 

(2 


-  I  Ml  C.(w)p(w)  -  9  ).  (3- 

k-0  K  ft  K  K 


Taking  partial  derivatives  and  equating  to  zero, 


3F(£) 

3p -  Jln(p*(ui)/7T(w))  +  p*(w) 


*(<*>) 

P*(w) 


1 

tt(w) 


n 

-  £  A.  C.  (w)  ■  0,  all  w  e  ft 

•  a  K  K 


(3-i 
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© 


and 


3P(p)  p 

■  I  Ck(a>)p*(w)  -  ©k  •  0;  k-0,1 . n. 


(3-66) 


Simplifying  (3-65)  and  (3-66), 


4n(p*(w)/ir(w))  -  £  X.C  (oj)  -  1,  all  u  e  Q  , 
k-0  K  K 


(3-67) 


o 


and 


i 


0 


0 


l  C^fcOpMw)  -  8kJ  k-0,1 . n.  (3-68) 

n 

In  multinomial  sampling  \  p(w)  -  1,  and  letting  C  (u)  -9,-1  for 

fl  Ok 

all  oj  e  satisfies  the  constraint.  Now  solving  (3-67)  for  the  p*(ui), 


—  1 

p*(u>)  -  e  ir(u>)exp  \  (X.C,  (w)),  all  u>  e  8, 

k-0  K  K 


and  letting  X^  -  X©  ”  1» 


p*(w)  -  exp(X^  +  X^C^(u)  +  X2C2(u>)  +  ...  +  XQCn(w))ir(a)), 

all  w  e  n  (3-69) 

where  the  X^  are  to  be  determined  to  satisfy  the  constraints,  (3-68). 

The  solution,  (3-69),  is  of  exponential  form.  Rewriting  (3-67), 


in(p*(w)/Tr(u>))  -  Xq  +  AjC^Ga)  +  X2C2(u>)  +  ...  +  A^C^oi),  (3-70) 


and  the  similarity  to  Birch's  log-linear  model  Is  apparent  [e ,g. ,  the 
r*s  model  given  in  (3-6)] .  Birch  (1963)  presented  his  model  to 
represent  the  factors  and  multiplicative  interactions  of  the  contingency 
table.  On  the  other  hand.  Rollback's  model  is  naturally  derived  based 
on  the  theory  of  minimum  discrimination'  information  relating  two  dis¬ 
tributions  over  the  same  set. 

By  the  convexity  property,  (3-60b) ,  Equation  (3-69)  will  indeed 
provide  a  minimum  I(jj:tt);  i.e.. 


I(j»*jir)  -  J  p*(ui)  4n(p*(m)/iT(«))  *  minder):  £  e  P).  (3-71) 

ft 


this  leads  to  another  significant  property  of  the  MDI  statistic  proved 
by  Kullback  (1959),  the  additive  property, 


:v)  »  l(p*sw)  +  !(£:£*),  *£  e  P. 


(3-72) 


As  explained  in  Section  3.3,  the  ultimate  goal  in  contingency 
table  analysis  of  detarmining  the  relationship  among  the  variables  can 
be  accomplished  by  using  hypothesis  tests  on  observed  tables  to  fit 
log-linear  models*  For  the  "independence"  hypotheses  the  fitted 
models  depend  on  some  set  of  observed  marginals.  Gokhale  and  Kullback 
(1979a)  define  these  type  of  problems  as  Internal  Constraints 
Problems  (ZCP).  for  the  "homogeneity"  hypotheses  no  set  of  marginals 


are  sufficient  to  described  the  model,  even  though,  under  product- 
multinomial  sampling,  the  fitted  table  may  conserve  some  of  the 
marginals.  Gokhale  and  Kullback  (1978)  classify  these  problems  into 
a  larger  set  called  External  Constraints  Problems  (ECP) .  Under  the 
MDI  analysis  each  of  these  type  problems  (ICP  and  ECP)  has  a  different 
approach.  This  section  will  only  present  the  ICP  approach. 

In  considering  the  observed  contingency  table  and  fitting  it  to 
a  log- linear  model,  it  is  more  convenient  to  work  in  the  frequency 
domain.  The  MDI  estimates  (m*)  of  the  expected  values  (m)  are 
m*(u>)  »  Np*(w)  for  multinomial  sampling.  For  the  ICP  problems  these 
MDI  estimates  (m*)  are  exactly  equal  to  the  maximum  likelihood  esti¬ 
mates  (m)  discussed  in  Section  3.4.  Iterative  procedures  for  obtain¬ 
ing  these  estimates,  similar  to  Equations  (3-54),  given  for  the  MLE 
and  the  no  second-order  interaction  hypothesis  in  a  three-way  table, 
were  developed  by  Ireland  and  Kullback  (1968a)  and  Ku  and  Kullback 
(1968). 

For  the  ICP  problems  the  MDI  statistic  (MDIS)  for  testing 
hypotheses  is 

2l(x:m*)  *2  £  x(u>)  Jln(x(co)/m*(u)) ) ,  (3-73) 

which  has  an  asymptotic  chi-square  distribution  under  the  null  hypo¬ 
thesis.  It  is  interesting  to  note  that  this  statistic  is  minus  twice 

the  likelihood  ratio  statistic,  whose  properties  have  been  empirically 

2 

compared  to  the  Pearson  X  and  other  statistics  in  some  hypothesis 
testing  situations  as  extensively  reviewed  in  Chapter  II. 


Working  now  in  the  frequency  domain  for  contingency  tables,  x 
is  the  ft*l  matrix  of  observed  cells,  x(to),  arranged  in  lexicographic 
order.  Let  ^  be  a  ft  x  (n+1)  "design  matrix"  of  rank  r+1  £  ft  (T'  *  C 
in  (3-61)).  The  linearly  independent,  n+1  columns  of  are 
T^(co)(l  _<  to  <_  ft)  for  each  i,  0  £  i  <_  n.  The  "design  matrix"  corre¬ 
sponds  to  the  fitted  observed  marginals  which,  in  turn,  correspond  to 
the  hypothesis  of  concern.  These  appropriate  observed  marginals  are 
the  constraints  as  specified  in  (3-61).  Therefore, 

T’m*  -  CN£*  -  N0  =  Cx  -  T'x.  (3-74) 

The  columns  of  _T  will  be  indicator  functions  for  the  required  fixed 
marginals.  The  T^(w)  will  be  one  or  zero  depending  on  whether  or  not 
the  cell  is  included  in  the  corresponding  marginal.  By  convention, 
for  multinomial  sampling  Tq(co)  ■  1  for  all  to,  and  the  total  sum  con¬ 
straint,  £  m*(to)  ■  £  x(to)  »  N,  is  satisfied, 
ft  (2 

Changing  notation  (t^  »  X^,  Ti  ■  X.^;  1  £  i  £  n)  and  rewriting 
(3-69)  with  m*(to)  *  Np*(to),  the  MDI  estimates  have  the  exponential  form 

m*(to)  -  exp (Tq  +  t1T1(u)  +  T2T2(w)  +  ...  +  x^T^ (to) ) Ntt (co)  , 

all  ci)  e  ft  .  (3-75) 

Summing  the  m*(to)  over  ft, 

[#)  ■  Nexp(TQ)  ^  exp(T1T1(w> +T2T2(to)  +  ... +TnTn(to))  it  (to)  -  N, 


and  solving  for  Tq , 

tq  -  -£n[£  exp^T^ui)  +  t2T2(co)  +  ...  +  TnTn(oj)  )tt(oj)  ) .  (3-76) 

(2 

The  remaining  parameters  (t^)  need  to  be  determined  so  that  the  marginal 
constraints  in  (3-74)  are  satisfied  for  the  resulting  estimates  (m*) 
of  (3-75). 

The  iterative  proportional  fitting  procedures,  discussed  in 
Section  3-4  for  MLE  and  given  in  Ireland  and  Kullback  (1968a)  and  Ku 
and  Kullback  (1968)  for  MDI  estimates,  cycles  through  the  marginals 
corresponding  to  the  constraints  of  (3-74)  until  the  desired  level  of 
accuracy  is  obtained.  The  distribution  'rr(u))  is  used  to  begin  the 
procedure.  The  procedure  allows  ir(co)  to  be  any  distribution  which  sat¬ 
isfies  some  of  the  marginal  constraints  of  (3-74)  but  no  others.  Any 
additional  marginal  constraints  satisfied  will  remain  at  the  end  of  the 
cycling  process,  and  the  MDI  estimates  (m*)  will  contain  additional 
exponential  parameters  in  (3-74)  [due  to  tt(w) ] ,  which  are  not  related 
to  the  Lagrangian  parameters  corresponding  to  the  hypothesis  of  con¬ 
cern.  It  is  convenient  to  let  tr(a>)  be  the  uniform  distribution,  1/0, 
for  multinomial  sampling  problems.  Then  tt(w)  satisfies  the  probability 

distribution  constraint,  j[  n(co)  ■  1.  Correspondingly,  the  beginning 

0 

cycle  estimates  are  N/Q  and  satisfy  the  sample  size  'constraint, 

-*• 

\  N/Q  *  N.  No  other  constraint  is  satisfied,  so  no  extraneous  para- 

0 

meters  will  be  introduced  into  (3-74). 

3.6,3  The  MDI  Log-Linear  Model  and  Hypothesis  Tests 

The  exponential  form  for  m*(w),  (3-75),  can  be  changed  to  a  log- 
linear  model  by  dividing  by  Ntt(w)  and  taking  logarithms, 


7*.:  : 


J,n(m*(a))/Nir(co))  -  tq  +  TjT^w)  +  t2T2(u))  +  ...  +  tM^Coj), 

all  w  e  0 


(3-77) 


In  matrix  notation 


An(m*/NTT)  -  T  T, 


(3-78) 


where  £n(m*/Nir)  is  the  0x1  matrix  with  terms  Jtn(m*(w)/N7r(u)))  in 
lexicographic  order,  and  _r'  -  (To*Tl,T2*  *  **  ,Tn^  *  ™-s  ®°del  can  be 
compared  to  Birch's  log-linear  model  discussed  in  Section  3.2. 

Consider  the  2x2  contingency  table  and  the  log-linear  model 
given  in  (3-15).  This  "complete"  model  in  matrix  notation  is 


£n(m)  ■  X  B, 


(3-79) 


where 


(ftn(m) ) '  ■  (in  ®u»&1  m21,2n  m22) , 


B/  -  (u',u1,u2,u12) , 


1  -1  ) 

1  -1  -] 


.  a* 


With  the  four  independent  parameters  and  only  four  cells  the  MLE 


and  MDI  estimates  will  exactly  equal  the  observed  values;  i.e., 

®ij  "  “ij  "  xij;  J*1*2* 


Therefore , 


in(x)  -  X  B.  (3- 

The  corresponding  "complete"  log-linear  model  in  the  MDI  parameteri¬ 
zation  from  (3-78)  using  ■  1/4  is 

fn(x/N/4)  »  jtn(m*/N/4)  «  T  T,  (3- 


where 

i'  ■  <VVW 

and 

Ilf 
10  0 
0  10 
0  0  ll, 

Note  that  the  first  three  columns  of  T  specify  constraints,  T'm*  ■ 

where  the  sample  size,  £  £  a*  •  £  £  x..  ■  N,  and  marginals, 

i  j  13  i  J  3 


«.**  wT*~_?V  »--  m  V  '.*^.  ■*.•  »'.►*•*' 


5  .  -. 

1  O 
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t: 


! 

i 

i 

! 


y  mf.  ■  )  x..  and  y  m*  '■  V  x. , ,  are  fixed.  The  fourth  column 
J  lj  J  ij  i  11  i  11 

requires  that  m*^  •  x^  and,  as  stated,  uniquely  determines  that 
"tj  '  xi 3-  i"l»2;  j«l,2.  Solving  for  the  T  parameters. 


o1 

o 

o 

An(x11/N/4)" 

« 

T  -  T  An(x/Nir)  - 

0  10-1 

An(x12/N/4) 

1  *  -\ 

0  0  1-1 

An(x21/N/4) 

f 

1  -1  -11 

_An(x22/N/4)^ 

iO 


r 


I 


An  xnn  - 


22 


A  (N/4) 
n 


An  x,  „  -  An  x 


‘12 


22 


An  x„,  -  An  x, 


(3-82) 


‘21 


22 


An  x,,  - 


11 


An  x^2  “  An  *21  +  2,0  *22* 


An  alternative  way  to  derive  this  parameterization  would  be  to 
use  the  same  starting  log-linear  model  as  Birch  but  to  apply  different 
constraints.  From  (3-6)  Birch's  model  with  T  parameters  would  be 


An(x1j/n7r1j)  ■  To  +  Ti(i)  +  T2( j)  +  X3(ij);  1-1,25  J-1,2‘ 


Now,  instead  of  applying  the  summation  constraints  on  the  parameters. 


let  t1(2)  -  t2(l)  -  tj(12)  -  t3(21)  -  t3(33)  -  0  and  tq  -  M*u/nir) . 


This  allows  for  the  following  parameterization, 


T1  "  Tl(l)’  T2  “  T2(2) *  X3  "  T3(ll)’ 


and  the  model  given  in  (3-81)  with  T  parameters  of  (3-82) 


The  t  parameters  of  (3-82)  can  be  compared  to  the  i 
of  (3-80).  From  (3-80) 


and 


6  -  X-1  in(X) 


1 

4 


1  1 

1  -1 

-1  1 

-1  -1 


"in  «  ' 

in  x12 

In  *21 

J 

>  *22. 

-|[2n  x^  +  in  *12  +  in  x21  +  in  x22] 


*Ito  *n  +  w  «12  -  »  *21  -  w  *22] 


■4  [in  xxi  -  in  x12  +  in  x21  -  in  x22] 


‘12 


■|[in  x^  -  in  x,9  -  in  x71  +  in  x99]. 


‘12 


21 


22 


Equating  (3-80)  and  (3-81) , 


X  i  -  T  t  -  in(N/4) 


parameters 


(3 


T  »  T  X  S  +  T  Jln(N/4) 


11 


1  0-1  1  1-1 

0  1  -1  1  -1  1 

-1-1  1  I  I  1  -1  -1 


-1  -1 


£n(N/4) 
An(N/4) 
£n(N/4) 
£n(N/4)  ; 


so  chat. 


“o  ■  U1  ■  u2  +  u12 

2("l  -  ul2) 

2(u2  -  uu) 


-  in(N/4) 


(3-8^ 


4u12‘ 


The  Interpretations  of  the  T  parameters  differ  somewhat  from 
those  of  the  u  parameters  discussed  in  Section  3.2.  The  T  parameters 
establish  the  i»2,  J*2  cell  as  a  comparison  cell  by  fixing  the  1*1, 
j*l  cell.  Tq  is  the  normalized  comparison  value.  Then  x^  measures  the 
log-ratio  between  being  in  category  one  of  variable  one  and  category 
two  of  variable  one  within  category  two  of  variable  two.  measures 
the  log-ratio  between  being  in  category  one  of  variable  two  and  category 
two  of  variable  two  within  category  two  of  variable  one.  x^  measures  a 
relation  similar  to  u^>  an  interaction  effect  between  x^  and  that 

is,  after  correcting  for  the  normalized  standard  Xq,  how  far  x^  and 
x.  differ  from  the  normalized  fixed  cell;  i.e., 
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t3  -  (An  x^  -  An  N/4)  “  T0  ”  Ti  "  T2 

-  An  x^  -  An  N/4  -  An  x22  +  N/4  “  xi2 

+  An  x22  -  An  x21  +  An  x22 

-  An  x11  -  An  x^2  -  An  x21  +  An  x22< 

Even  Chough  Che  single  effecc  u  and  x  paramecers  differ,  Che  relaClons 
In  (3-84)  indicate  chat,  excepC  for  a  conscanC,  u^2  and  measure  Che 
same  "InCeracClon"  relaClon  among  Che  cell  values.  Therefore,  as 
mighc  be  expecCed,  Che  Independence  hypothesis  cescing  procedures  for 
Che  Cwo  models  are  similar. 

Recall  chac  in  Birch's  log-linear  model  Che  classic  hypoChesls 
of  Independence  in  a  2  *2  Cable  viCh  mulCinomial  sampling  is  equlvalenC 
Co  cescing  Che  hypoChesls,  HqJ  u^2  *  0.  In  Kullback's  model  che  pro¬ 
cedure  is  Co  form  che  ^  macrix  based  on  Che  fixed  sample  size  and 
marginals  corresponding  Co  che  null  hypoChesls.  These  marginals  are 
Che  same  sufflclenC  statistics  required  Co  find  Che  MDI  esClmaCes  (or 
equlvalenC  MLE)  in  Che  iceracive  proporcional  fleeing  procedure.  For 
Che  above  hypoChesls  cescing  sicuaclon  ehese  are  Che  marginals  m^  and 
maj.  Recalling  also  Chac  Che  columns  muse  be  lndependenc,  noCe  chac 
only  one  of  Che  marginals  from  each  sec  of  cwo  is  required;  i.e.,  m^ 
or  m2>  and  m^  or  m>2.  For  a  general  r*s  Cable,  r-1  marginals  from 
m^#  and  s-1  marginals  from  m^  would  be  required.  Kullback  always 
selecCs  che  lowest  ordered  marginals;  so  for  che  2*2  case 


corresponding  to  the  constraints  from  (3-74),  £  £  m*  ■  £  £  x  , 

ij  J  ij  3 

I  “  I  and  £  -  £  x^.  The  log-linear  model  from  (3-78) 

with  tt^.  ■  1/4  is 


£n(m*1/N/4) 

JLn(m*2./N/4) 


Hn(m^1/N/4) 

Jln(m^2/N/4) 


Solving  for  the  T  parameters. 


T0  +  T1  +  t2 


T0  +  Tl 


T0  +T2 


Tq  ■  Jin  m^2  ”  An(N/4) 


-  Jin  m*2  -  2-ti  m*2 


t2  "  Jin  m*L  -  Jin  m^; 


leaving 


Jln(m*1/N/4)  “  tq  +  Ti  +  T2 


Jin  m*2  “  Jln(N/4)  +  Jin  m*2 
+  In  m^  -  £n  m*2> 


Jin  m*2 


St*  it i  ..*»  — '» — *». 
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which  Implies  that 

Jin  is*^  -  Jin  m*2  -  Jin  m*^  +  Jin  m*2  -  0,  (3-87) 

or  as  might  have  been  expected,  x^  ■  0.  The  complete  model,  (3-81), 
allows  the  x  parameters,  (3-82),  to  be  calculated  directly  based  on  the 
observed  values.  The  independence  model,  (3-85),  uses  the  MDI  estimates, 
which  have  been  calculated  under  independence  to  require  that  x^  be 
equal  to  zero  as  determined  in  (3-87).  Thus,  the  independence  hypothesis 
is  equivalent  to 


Hq:  x3  -  0  in  the  Kullback  model. 


or 


H0;  u^  ■  0  in  the  Birch  model. 

Equation  (3-87)  can  easily  be  verified  by  using  the  MDI  esti¬ 
mates  (or  MLE) ,  ra*  ■  x,  x  ./N: 

ij  i* 

Jln(x^x^/N)  -  Jln(x^tx>2/N)  -  £n(x2#xt^/N) 

+  Jln(x2,x,2^^ 

■  Jin  x,  +  Jin  x  ,  -  &n  N  -  £n  x-  -  Jin  x  - 
1»  »1  l»  *2 

+  Jin  N  -  Jin  x»  -  Jin  x  ..  +  Jin  N  +  Jin  x9 
2*  *1  2* 

+  Jin  x  «  -  Jin  N 
•2 


0 
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Consider  now  the  three-way  2x2x2  table  under  multinomial 
sampling.  The  complete  Kullback  log-linear  model  is 


An (X/ Nit)  -  It; 


(3-88) 


where  An(X/Nir)  is  an  8x1  matrix  whose  values  are  AnCx^^^/Nir^^) ; 
i-1,2;  j-1,2;  k-1,2; 


T*  -  (L  T1  Tj  Tk  Tij  Tlk  Tjk  T^k)  • 


11111111 
11101000 
11010100 
11000000 
10110010 
10100000 
10010000 
1000000  0. 


Note  the  change  in  the  T  notation.  This  new  notation  from  Gokhale  and 
Kullback  (1978)  allows  extension  to  any  size  table.  L  is  the  normal¬ 
izing  term  (Tq) .  The  number  of  subscripts  corresponds  to  the  type 
effect  (one  for  main  effect,  two  for  first-order  interaction,  and 
three  for  second-order  interaction) .  The  superscripts  designate  the 
variables  in  the  effect.  The  subscripts  designate  the  category  of 
the  corresponding  superscript  variables.  Solving  the  model  for  the 


1  w  VjV  v  _\w .v.v.y  v  '.\v. 


parameters  (letting  it  ^  -  1/8) , 


L  * 


ll 

r[i 

Tlk 

T11 

11 

TiJk 

Tlll 


An(x222/N/8) 

S'n(x122/x222) 

to(x212/x222) 

£n(x221/x222) 

Jln(x112/x222  •  x222/x122  .  x222/x212)  -  in(xu2x222/xl22x212) 
2,n(xi2i/x222  *  X222^X122  *  X222/,X221^  “  *n^x121x222^xl22x221) 
J,n^x211^x222  *  X222^X212  *  X222^X221)  *  *'n^x211X222/,X212X221^ 
in(xlll/x222  *  X222/x221  * x222/x212  *  x222/x122  *  x122x212/ 
X112X222  *  X122X221'/x121X222  *  X212X221/x211X222) 


)ln(xlllx122x212x221/x112x121x211x222)* 


(3- 


The  interpretation  of  the  x  parameters  follows  from  the  2x2 

model.  Note  that  the  *222  term  has  been  established  as  the  comparison 

or  normalization  term.  The  main  effects  are  calculated  as  log-ratios 

within  the  second  levels  of  the  other  variables.  The  first-order  inter 

action  terms  are  log-ratios  within  the  second  level  of  the  other  vari- 

ilk 

able.  The  remaining  term,  x^-^,  is  then  a  measure  of  the  difference 
in  the  value  of  the  1*1,  j*l,  k*l  cell  and  what  is  predicted  by  the 
terms  which  have  been  calculated  within  the  1*2,  j*2,  k*2  levels. 

Note  the  relation  to  the  Uj^  term  of  Birch's  log  linear  model  given 


in  (3-34), 


rIll  is  lndee^  a  oeasure  of  second-order  interaction  under  the  classic 


Bartlett  definition. 


The  no  second-order  Interaction  model  is 


Jtn(n*/NTT)  -  It, 


where  in(a*/Wr)  is  an  8x1  matrix  whose  values  are  in(mj  , 
i“l»2;  j"l»2j  k-1 , 2 ;  and  are  the  MDI  estimates  calculated  by 
iterative  proportional  fitting; 

t' -  ig)i 


1111111 
1110  10  0 
1  1  O'  1  0  1  0 

1  1  0  0  0  0  0 

10  110  0  1 
1  0  1  0  0  0  0 

1  0  0  1  0  0  0 

1  0  0  0  0  0  0 


Thn  T  tana  are  the  same  as  in  (3-89)  with  replacing  a*^k  for  all 
i,  j »  and  k.  has  then  bean  forced  to  zero.  Therefore,  as  in 
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Birch's  model  (Hq!  u^23  *  0)»  the  nu^  hypothesis  of  no  second-order 

iik 

interaction  is  equivalent  to  Hq!  tiii  *  0  in  Kullback's  log-linear 
model,  (3-88). 

3.6.4  Asymptotic  Covariances 

With  respect  to  Kullback's  MDI,  some  discussion  of  the  asympto¬ 
tic  covariance  structure  will  help  to  relate  the  other  contingency 
table  methodologies.  Kullback  (1970)  derives  a  general  formula  for 
finding  the  asymptotic  covariance  structure  of  any  ICP  based  on  the 
relationship  of  the  "moment"  parameters  (observed  marginals  and  observed 
cells)  and  the  "natural"  parameters  (x  parameters  in  the  MDI  log-linear 
model).  Given  the  MDI  model  [(3-78),  where  m*  »  x  for  complete  models] 

in  matrix  notation  with  ir(w)  -  1/Q  (for  all  w  £  Q) ,  let  D  be  a 

x 

diagonal  matrix  with  diagonal  entries,  x(to)  ,  in  lexicographic  order. 

Find  the  matrix 


111 

—12 

-21 

-22 

-  T'D  T  , 
- x— 


(3-91) 


where  is  a  1><1  matrix  corresponding  to  the  normalizing  constraint, 

£  m*(w)  *  £  x(w)  ■  N,  So,  is  a  n*n  matrix,  and  ,  -  Si.  are  l*n 
fl  fl 

matrices.  Then  the  estimate  of  the  asymptotic  convariance  matrix  of 
the  moment  parameters  [those  specified  by  _T  in  the  constraints  of 
(3-74),  T'm*  -  T'x]  is 


-22.1  “  -22  “  -21-11-12* 


(3-92) 


The  corresponding  estimate  of  the  covariance  matrix  of  the  T  parame¬ 
ters  of  (3-78)  is  the  inverse  matrix 

To  demonstrate  the  procedure,  consider  the  2><2  table  and  com 
plete  model,  (3-81).  Following  the  above  procedure. 


be  verified  since  the  x^  are  observations  from  a  multinomial  distri¬ 
bution.  These  observations  can  be  used  to  estimate  the  p^ ,  = 

xij/N,  and  to  calculate  each  term  of  j»22  from  variance/covariance 
formulas  for  the  multinomial.  For  example. 


V(x1#)  -  V(x13L  +  x12)  -  V(x11)  +  V(x12)  +  2COV(x1;l,x12) 


N  PUC1-PU>  +  N  P12(l-P12)  +  2(-N  PnP12) 

N  X11(X12  +  X21  +  X22J  +  N  X12(xll  +  X21  +  X22J  "  N  2xllX12 
-  *n(x21  +  x22)  +  $  xl2(x21  +  X22} 


0£1.x2.)/N* 


which  is  the  (1,1)  term  in  (3-94).  Other  terms  are  similarly  verified. 

A  more  general  approach  for  calculating  asymptotic  convariance 
matrices  for  asymptotic  normal  parameters  is  the  well-known  "Delta 
Method".  Bishop,  Fienberg,  and  Holland  (1975)  provide  theorems 
(Theorems  14.6-1,  2,  3,  4)  for  its  use  in  both  the  single  variate  and 
multivariate  cases.  As  applied  to  contingency  tables,  to  find  the  es¬ 
timated  asymptotic  covariance  of  a  matrix  function  (_f(x)  of  size  n  *  1) 
of  the  observations,  the  function  must  be  differentiable  with  respect 


to  x.  This  derivative  is 


Ill 


Next,  the  estimated  covariance  matrix, V(x) ,  of  the  observations  from 
the  appropriate  sampling  distribution  must  be  found.  The  estimated 
asymptotic  covariance  matrix  for  f(x)  is  then 


V(f(x))  «  (3f/3x)  V(x) (3f/3x) ' . 


(3-96) 


For  the  2x2  table  and  complete  model,  (3-81), 


(  f  (x) ) »  -  (xj  .x  j.x^); 


so  that. 


110  0' 
10  10 
1  0  0  0  , 


which  is  the  T  matrix  minus  the  top  row  corresponding  to  the  sample 
size  constraint.  From  the  multinomial  distribution,  the  estimated 
covariance' matrix  for  x  is 


V(x)  -5 


!tu(8-xu) 

”x12xll 

-X21X11 

”X22X11 

“X11X12 

x12(N-x12) 

"X21X12 

"X22X12 

"xllx21 

“X12X21 

x2i(N-x2i) 

”X22X21 

"X11X22 

”x12x22 

”x21x22 

x22(N-x 

(3-97) 


Then  from  (3-96) 
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V( f (x) )  -  i 


X1.X2* 

X11X22“X12X21 

xllx2. 

X11X22“X12  21 

x.lx.2 

xllX-2 

(3-98) 

X11X2 • 

XHX.2 

X11(N"X11) 

• 

(1,2)  and  (2,1) 

of  ^22  1  8iven  in 

(3-94), 

xl.x.l 

X11  N  "  ^Xll ^Xll +  X12  +  X21  +  X22^  “  ^*11  +  X12^  ^Xll +  X22^ ^ 


IX11X22-X12X21]/N* 


Hence  1  and  are  equivalent. 

Bishop,  Fienberg,  and  Holland  (1975)  provide  a  theorem  (14.6-4) 
for  the  "Delta  Method"  directly  applicable  to  log-linear  functions, 
^(x),  under  multinomial  sampling.  If  f .  (x)  ■  £  C.  ((*))  in  x(w) ; 

“  1  n 

i»l,2,...,n;  then  the  terms  of  the  9f/9x  matrix  are 


9fi 

SK7*  ci <“>/*<“>• 


In  matrix  notation 


3f/3x  »  C  D^1  , 


(3-99) 


where  (£)it0  ■  (C^w)).  Then  the  theorem  supplies  the  estimated  asymp¬ 
totic  covariance  matrix  of  _f  (x)  , 


,  '»  *V  «4  -»  -  -•  .  •  .  »  v  ■»  «  '  .*  ,*  -  «-  »  »'  «.  ■  ,  •■,  '  '  * 


V(jE(x))  -  C  D" 1  C’  -  (e  C’ )  ’e  C’ , 


(3-100) 


where  _e  *  (1,1,...  ,1)  is  a  1*Q  vector. 

This  procedure  can  be  applied  to  find  the  estimated  asymptotic 
covariance  of  the  taus  in  the  2x2  complete  model  under  multinomial 
sampling.  These  taus,  (t^t^t^),  as  given  in  (3-82)  are  linear  com¬ 
binations  of  the  logarithms  of  jc.  Forming  the  appropriate  matrices. 


0  10-1 
0  0  1-1 
1-1-1  1 


e  -  (1,1, l,.l) 


From  (3-100) 


_f (x)  ■  £  £n(x) . 


-(T1’T2,T3>  "  I(i(i)) 


1 

1  1 

x12  x22 

x22 

X12 

x22 

1 

1  +  .  1 

_  __1_ 

_ 1_ 

x22 

X21  x22 

X21 

x22 

_  _1 _ 1_ 

_  JL _ 1_ 

1  ^ 

A  .  : 

x12  x22 

X21  x22 

xli  X12  x; 

(3-101) 


With  the  above  procedure  the  estimate  of  the  asymptotic  covari¬ 
ance  matrix  for  the  u  terms  of  Birch's  log-linear  model  can  also  be 
calculated.  For  the  2  *  2  table,  from  (3-83)  for  (u^,u2,u^2) 


C  - 


1  -1  -1 

-1  1  -1 

-1  -1  1 


» 


and  using  (3-100) , 

V(Ul,u2,ul2) 


1 

,1,1,1 

1 

1 

_ 1__ 

h_l_ 

_1 _ 1  ,  1 

1 

X11 

X12  X21  X22 

X11 

X12 

X21 

X22 

X11  X12  X21 

x22 

1 

_  _ L.+JL 

1 

1 

1  ,  1 _ 1_ 

1 

X11 

x12  X21  x22 

X11 

X12 

X21 

X22 

X11  x12  X21 

x22 

1 

_ L. 

_L.+_i_ 

1 

1 

+_1_ 

_xn 

x12  X21  x22 

X11  X12 

X21  x22 

X11  X12  X21 

x22_ 

(3-102) 


Note  that  £(t.j)  *  16V(u12), which  is  consistent  with  the  relation  in 
(3-84).  The  other  relations  for  and  t2  in  (3-84)  could  also  easily 


be  verified 
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3.6.5  Asymptotic  Comparisons 

Using  these  covariances,  Kullback  has  derived  various  estimators 
for  the  MDX  statistics.  These  are  derived  in  general  terms  in  Appendix 
A  and  given  as  Equations  (A-21)  and  (A-22).  As  an  example,  consider 
again  the  2x2  table  and  the  ICP  problem  of  the  independence  hypothesis 
under  multinomial  sampling.  The  appropriate  hypothesis  testing 
statistic  is  given  in  Equation  (3-73),  2I(x,m*).  Here  the  observed 
and  estimated  tables  have  reversed  roles.  The  partitioning  described 
in  Appendix  A,  (A-18) ,  is 


0*  -  (0*,0|)  -  ((x1#,x#1),xi;L) 

-<9a»V  “  ((ol.»m*l*)’mll)’  (3"103) 

where  9*  ■  9..  The  £  matrix  is  the  estimated  covariance  matrix  of  the 
—A  —A  — 

moment  parameters  (m*  ,m*^,m*^),  which  is  S^2  1  °f  (3-94)  with  the  MDI 
estimates,  m*^ ,  in  place  of  the  x^ ,  and  1  is  S^2  ^  of  (3-95)  with 
m*j  replacing  the  x^ .  Noting  that  j)*  and  6^  are  1  x  1  matrices  and 
9*  ■  9^,  from  Equation  (A-16)  and  (A-22), 


2I(£*:tt)  2  ( 9*-6) '  £"1(8*-6) 


<sj-v 


Then,  £~^  A  must  be  the  (3,3)  term  of 
and  the  approximation  is 


it  T  jl  T  ff  ~  * 

®11  m12  m21  m22 


» 
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2I(x:m*) 


1 


Recalling  chat  the  marginals  of  the  observed  and  MDI  tables  are 


K11  ~all  *  xl»x12  - 


"*2  ‘  x12 


or 


(Xi2-m*2)' 


X11  *  “!l 


X.1'X21  *  <*-l-m21)  *  ”21  -  X21 


or 


2 

# 


*11**B11 


i*  - 


m 


12 


-  x 


12 


-  m: 


22 


^x.2~x22^ 


X22  '  m22 


or 


^*11  ”  mll>  "  (x22“®22)  ' 


(3-104) 

equal, 


Substituting  Into  (3-104), 
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2l(x:m*)  s:  (xn +  (xi2  ~  a*2)  2/,m12 

+  (x2.  “  m2.)2/®2.  +  (x22"m22)2/®22 

*  (Xij  (3-105) 

For  Che  ICP  problems  Che  MDI  esCimaCes,  m*  ^ ,  are  equal  Co  Che  MLE; 

2 

Cherefore,  Che  approximaCion  is  Che  Pearson  X  wich  MLE  for  Che  expecced 
values . 

If  now  2l(m*:x)  is  esCimaCed  using  Che  same  procedure,  Che  co- 
variance  maCrices,  £  and  £  \  are  in  Cerms  of  Che  observed  cable,  and 
Equaclon  (A-22)  becomes 


2I(m*:x)  a  (x^ 


!(— 

\X11 


(3-106) 


Using  Che  same  subsCiCuCions  as  above,  Che  righc-hand  side  of  (3-106) 
is  equal  Co 


l  l  <* 

i  J 


ij 


-a*,)  /x.j. 


2 

This  is  Neyman's  modified  chi-square  scaciscic,  Y  ,  wich  che  MLE  for 
esCimaCes  of  Che  expecCed  values.  This  scaciscic  is  useful  in  ECP 
Cype  problems. 


If  (A-21)  is  now  used 


where  Tg  is  T^.  From  (3-82),  is  equal  to  (in  x^  -  in  x^2  - 


in  x2^  +  in  x22) »  so  (A-21)  is 


2I(jc:m*)  «  (in  x^^  -  in  x^2  -in  x^ 


+  Jtn  x,.)2(— +—+—+— )_1.  (3-107) 

22  \n  xn  x22 


In  the  next  section,  the  right-hand  side  of  (3-107)  will  be  shown  to 
be  equal  to  the  GSK  statistic  for  the  2x2  independence  hypothesis 
test. 

All  four  statistics  (Pearson,  Neyman,  Kullback,  and  GSK)  are 
asymptotically  related  through  their  asymptotic  covariance  structure. 
The  Pearson  and  GSK  statistics  are  asymptotically  equivalent  to  the 
Kullback  ICP  statistic  (log-likelihood  ratio  statistic),  and  the  Neyman 
statistic  is  asymptotically  equivalent  to  the  Kullback  ECP  statistic. 

3.7  Grizzle.  S tanner,  and  Koch  (GSK) 

The  Grizzle,  Starmer,  and  Koch  (GSK)  approach  to  the  analysis 
of  contingency  tables  is  based  on  the  methodology  of  weighted  least 
squares.  The  approach  has  gained  wlde-acceptance  and  has  been  applied 
to  many  varied  types  of  problems.  Probably  its  most  attractive  feature 
is  that  it  can  be  used  with  a  number  of  different  assumed  models  to 
Include  both  the  linear  and  log-linear  models.  First,  the  general 
procedure  will  be  discussed,  then  the  procedure  with  an  assumed  log- 


linear  model  will  be  described 


3.7.1  General  Derivation 

In  order  to  better  compare  the  contingency  table  methodologies. 
Rollback's  notation  will  be  used  in  place  of  the  standard  GSR  nota¬ 
tion.  Let 


(p(l),p(2),. ..,?(£)))  (3-108) 

be  the  l*ft  vector  of  table  probabilities  in  lexicographic  order.  Let 
p(u>)  be  sample  estimates  of  these  probabilities.  For  example,  for 
multinomial  sampling 


£(u>)  ■  x(w)/N,  (3-109) 

where  x(u>)  are  the  observed  values  and  N  is  the  total  sample  size.  Then 


£’  -  (p(l),p(2), ...,£(£})) 

x'  -  (x(l),x(2) ,...,x(ft)). 


(3-110) 


Rnowing  that  the  asymptotic  covariance  of  j>  is  V(£)  -  V(£) ,  where  the 
form  of  V(£)  is  determined  from  the  sampling  model,  an  estimate  of 
V(£)  can  be  obtained  by  replacing  p(w)  by  {J(w)  in  J/(£) .  Let  this 
estimate  be  V(£) .  Let  fm(£)  (m»l,2, . . . ,n)  be  a  sequence  of  functions 
of  the  elements  of  £  that  have  partial  derivatives  with  respect  to  the 
p(w)  up  to  second  order.  Let 


120 


f^CjS)  *  evaluated  at  j>  =  jS 


[F(£)]'  -  [f1(£),f2(p) . fn(£)] 


F* 


H 

S 


[F(£)]'  • 

r 

3p(ui) 

H  1(£)H' 


[«!<£>. f2(® . fn(£)] 

p(w)  -  p(t*))j  (nxfi) 

(n  x  n) 


(3-111) 


From  the  "Delta  Method"  described  In  Section  3.6  and  Equation  (3-96), 
j>  Is  the  estimate  of  the  covariance  matrix  of  _F(£) .  If  £(£)  is  a 
linear  function  of  j>,  then  £  is  exact  in  the  sense  that  it  is  indepen¬ 
dent  of  sample  size.  If  F(jjO  is  a  non-linear  function  of  £  then  S[  is 
the  estimate  of  the  asymptotic  covariance  matrix. 

Next,  assuming 


F(£)  «X£,  (3-112) 

where  X  is  a  known  (nxd)  design  matrix  (analogous  to  Kullback's  £, 
Equation  (3-61)  and  (3-69),  or  Equation  (3-78)  J  and  £  is  a  (d  x  i) 
vector  of  unknown  parameters  (analogous  to  Kullback's  jr,  Equation 
(3-78)],  the  weighted  least  squares  method  determines  that  a  best 
asymptotic  normal  [BAN,  Neyman  (1949)]  estimate  of  £  is  £,  where  Jb  is 
the  vector  which  minimizes 

-  (F-X  b)'S-1(F-X  b). 


(3-113) 


This  weighted  least  squares  estimator  is 


b  -  (X,l"1X)_1X,l“1F.  (3-114) 

2 

The  statistic,  X^,  in  (3-113)  is  then  a  test  on  the  fit  of  the  model, 
(3-112),  and  has  an  asymptotic  chi-square  distribution  with  n-d  degrees 
of  freedom  under  the  null  hypothesis. 

If  the  model  does  fit,  a  test  can  be  performed  on  a  linear 
hypothesis  of  J3, 


H0:  K  B  -  0,  K  is  (k  x  d) , 


(3-115) 


with  the  statistic 


G2  -  (Kb)' (K(X?_S_1X) _1K' )-1K  b,  ( 3-116) 

which  under  the  null  hypothesis  has  an  asymptotic  chi-square  distribu¬ 
tion  with  k  degrees  of  freedom. 

3.7.2  Log-Linear  Model  and  Hypothesis  Tests 

Now  suppose  that  the  structure  of  _F(£)  is  log-linear;  i.e., 

F(2)-C(in£),  (3-117) 

where  C  is  (n*fi).  Then  from  (3-111) 


(3-118) 
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where  _D^  Is  a  diagonal  matrix  with  £  along  its  diagonal.  From  the 
"Delta  Method"  of  Section  3.6  and  (3-111), 

*C’ ,  (3-119) 

£  £- 

the  estimated  asymptotic  covariance  of  F(£).  Under  multinomial  sampling, 
from  (3-100)  this  becomes 

S  -  If1  [C  D_1C ’  -  (e  C’ ) » e  C’  ] .  ( 3-120) 

£ 

With  S  from  (3-119)  or  (3-120) ,  hypothesis  tests  are  performed  using 
through  (3-116) .  Covariance  formulas  for  b  and  F  are 

V(b)  -  (X’rtr1  (3-121) 

1(F)  *  XU'S"1*)’1}?*.  (3-122) 


As  an  example,  consider  the  2X  2  table  under  multinomial  sampling 
and  the  no  interaction  hypothesis.  The  first  step  is  to  choose  an 
appropriate  log-linear  model.  Consider  Birch's  complete  log-linear 
model  with  the  parameterization  of  (3-15)  and  formulas  in 

(3-83).  Forming  Equation  (3-117), 


F(£)  ■  C  £n  p. 


(3-123) 
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where 


and 


(in£)'  -  (in  pu,in  p12,in  p21,in  P22)  • 


From  (3-112)  the  model  is 


F(£)  -  C  in  £  «  X  8, 


(3-124) 


where 


X 


—3 


» 


the  third-order  identity  matrix,  and 


£'  *  (u1>u2»ui2^* 


Next,  finding  the  estimated  asymptotic  covariance  matrix  of  _F(j>)  from 
(3-119)  or  (3-120), 


1 


Since  ■  x^j/N,  (3-125)  is  identical  to  (3-102),  the  asymptotic 
covariance  for  (u^j^.u^)*  This  should  be  expected  since  (F(j>)) '  » 
(u^.Uj.u^)  from  (3-124).  For  this  log-linear  model  the  goodness-of- 
fit  test  is  inappropriate  since  this  model  is  complete,  and  from 
(3-114) 

b  -  (X,S_1X)_1rs'1F 

-  s  S~ll  -  F  -  F(p) ; 
so  that,  from  (3-113)  with  X  -  _I  , 

xj  -  (F  -  X  b)  'S'^F  -  X  b)  -  0. 

In  Section  3.3  it  was  shown  that  the  independence  hypothesis  test  was 
equivalent  to  u^2  ■  0.  In  the  GSK  notation  of  (3-115) 

H0:  K  8  •  0, 

where 

K  -  (0,0,1). 


The  GSK  statistic  is  then  calculated  from  (3-116) 


(3-126) 


[^(£n  Pn-*n  Pi2-Jln  P21+Jln  ?22 
N(in  plx  -  in  p12  -  in  pn  -  in  p22) 


J 


Equation  (3-107)  showed  (3-126)  to  be  an  approximation  for  the 
MDI  statistic,  2l(m*:x),  as  derived  from  the  Kullback  log-linear  model. 
Starting  with  Kullback's  log-linear  model,  (3-81),  and  the  T-,  t2>  and 
parameters  derived  in  (3-82),  (3-117)  can  be  formed  as 


_F(j>)  ■  C  in  p. 


(3-127) 


where 


C 


1 

0 


-1 


0 

1 


-1 


From  (3-112)  the  model  is 


F(£)  -  X  8, 


(3-128) 


The  estimated  asymptotic  covariance  matrix,  £,  is  then  the  covariance 
matrix  of  the  T  parameters,  given  in  (3-101)  in  terms  of  the  observa¬ 
tions,  j •  Writing  J3  in  terms  of  the  probability  estimates,  p^. 


1  i  1  1 

^12  ^22  ^22 


S 


_ 1 _ 1_ 

•  ^12  ^22 


1,1,1 
$12  $ 21 


(3-129) 


Knowing  that  *  0  corresponds  to  the  independence  hypothesis,  (3-115) 
is  again 


(3-130) 


with 


K  -  (0,0,1). 


The  GSK  statistic  is  then  calculated  from  (3-116) , 


G  -  [An  p.^-Jln  P12-Jln  p-.+An  p,J  t  ^  ^  +  + 1 

11  12  21  22  N  Pil  Pl2  P21  P22 


-  N(An  Pu-2-n  P12  -  An  P21  +  *n  P22)  /(^u  +?]^  +  ^21  +  * 


the  exact  Equation  (3-126). 

The  procedure  is  easily  extended  to  2x2x2  tables  and  the  no 
second-order  interaction  hypothesis.  Consider  Birch's  log-linear  model 
(3-32),  and  the  u  parameters  in  (3-33).  For  the  2x2  table,  only  the 
parameter(s)  corresponding  to  the  hypothesis  of  concern  needed  to  be 
considered.  This  is  true  for  any  hypothesis  testing  situation.  In 
(3-117) ,  F(£)  -  C  in  £,  let 


C  -  4(1, -1, -1,1, -1,1, 1,-1) 


(jin  p)  *  (In  Pm*^n  Pii2'^n  ^121* Pl22'^’**  ^211'^^  P212'^’^  ^221' 


An  p222^* 


In  (3-112),  F(£)  -  X  S,  let 


X  -  I,  and  8.  -  (u^-j)  • 


0\  "  * 


From  (3-120) 


Then,  for  the  GSK  hypothesis  of  (3-115),  K6.*  0,  let 


K  -  (1). 


The  GSK  statistic  from  (3-116)  is  then 


.2  _  (fapm-tnPu2-top121-fi.np122-2.p211^.p212-Hnp221-fap222) 

'  "  1,1  1  1  1  1  1  1 


X  ,  i.  .  X  ,  X 

Pm  Pn9  Pin  Pi ->9 


111  *il2  ‘'121  ‘'122  *'211  *xl2 


P,n  P, 


221  ^222 


(3-131) 


CHAPTER  IV 


EXPERIMENTAL  DESIGN 

The  most  important  aspect  of  any  study  of  this  nature  is  the 
design  of  the  experiment.  This  design  may  include  several  factors, 
such  as  the  selection  of  parameters  Co  measure,  the  selection  of 
independent  parameters  to  control,  the  settings  of  these  independent 
parameters,  the  procedure  to  use  to  measure  the  dependent  parameters, 
and  the  method  to  validate  the  procedure.  In  making  these  decisions, 
careful  consideration  must  be  given  to  the  overall  purpose  of  the  study 
and  the  expected  results  and  possible  conclusions.  Two  other  basic 
considerations,  which  often  come  in  conflict  and  often  determine  the 
range  for  the  selection  of  factors  above,  are  the  desired  level  of 
statistical  accuracy  and  the  constraint  of  cost. 

The  basic  procedure  used  in  this  study  is  a  Monte  Carlo  simulation 
to  approximate  exact  significance  levels  for  the  statistics  used  in  the 
analysis  of  contingency  tables.  With  consideration  of  the  above  ideas, 
this  chapter  discusses  the  selection  of  this  dependent  parameter  as  well 
as  the  selection  of  the  independent  parameters,  the  design  of  the  Monte 
Carlo  simulation,  the  design  of  an  exact  program,  and  the  validation 
of  the  Monte  Carlo  procedure. 

4,1  Selection  of  Parameters 

The  first  step  in  the  design  of  an  experiment  is  usually  to 
determine  the  parameters  of  concern.  In  this  study  under  the  null  hypo- 


thesis,  the  dependent  or  measurable  parameter  will  be  the  exact  level 
of  significance  of  the  test  statistics.  The  independent  parameters  will 
be  the  variables  of  the  underlying  sampling  distribution.  This  section 
will  discuss  the  selection  of  these  parameters,  a  problem  associated 
with  the  measure  of  the  dependent  parameter,  and  the  selection  of  the 
specific  values  for  the  independent  parameters. 

4.1.1  Exact  Levels  of  Significance 

As  stated  in  the  Introduction,  "The  primary  purpose  of  this  study 
is  to  investigate  the  robustness  characteristics  of  these  statistics 
with  respect  to  small  expected  values  as  the  size  of  the  table  increases. 
As  discussed  in  Chapter  III,  the  use  of  these  statistics  is  in  hypo¬ 
thesis  testing  in  order  to  fit  an  appropriate  model  to  the  data  and 
determine  the  relationships  among,  the  variables.  In  any  hypothesis 
testing  situation  the  procedure  is  subject  to  two  types  of  error:  type 
I  error,  related  to  the  rejection  of  the  null  hypothesis  when  it  is  true 
and  type  II  error,  related  to  the  acceptance  of  the  null  hypothesis  when 
it  is  false.  Usually,  the  type  I  error  is  controlled  in  that  the 
probability  of  this  error  (a)  is  preselected.  In  fact,  this  preselected 
a,  "level  of  significance",  becomes  the  basis  for  rejecting  or  accepting 
the  null  hypothesis.  The  type  II  error  is  uncontrolled  in  the  sense 
that  the  analyst  has  little  or  no  knowledge  of  the  actual  relationship 

of  the  hypothesis.  Therefore,  the  probability  of  a  type  II  error  (8) 

i 

cannot  be  calculated  specifically.  However,  in  many  hypothesis  testing 
situations  the  test  can  be  designed  so  that  the  analyst  is  "protected", 
within  a  designated  probability,  against  making  a  type  II  error  given 


a  specified  alternative  hypothesis.  This  forms  the  basis  for  the 


V.' 


operating  characteristic  curves  or  power  (1-8)  curves  available  for 
most  parametric  tests. 

With  respect  to  contingency  table  analysis  and  asymptotic  chi- 
square  statistics,  as  Cochran  (1952)  states,  "there  has  been  little 
demand  for  this  from  applications,  because  the  test  is  most  commonly 
used  when  we  do  not  have  a  clear-cut  alternative  in  mind,  and  are  not 
in  a  position  to  make  computations  of  the  power."  The  work  that  has 
been  done  with  respect  to  type  II  error  or  power  has  been  very  limited, 
usually  only  with  2x2  tables,  with  assumptions  on  the  underlying  dis¬ 
tributions,  and  with  results  presented  only  in  the  asymptotic  sense 
[e.g.,  Harkness  and  Katz  (1964),  Meng  and  Chapman  (1966),  and  Nathan 
(1972)].  Under  more  general  alternative  hypotheses,  the  problem  with 
respect  to  larger  contingency  tables  with  small  expected  values  seems 
Intractable. 

Reconsidering  the  type  I  error,  even  though  it  is  controlled  by 
the  preselection  of  a  nominal  level  of  significance  (a) ,  in  the  use  of 
these  asymptotic  chi-square  statistics  the  "exact"  level  of  significance 
(ae)  is  dependent  on  how  well  the  chi-squared  distribution  approximates 
the  exact  distribution  of  the  statistic.  This  means  that  in  the  use  of 
these  statistics  there  is  some  "uncontrolled'’  error  with  respect  to  the 
type  I  error.  This  "uncontrolled"  error  is  the  difference  between  the 
"nominal"  and  "exact"  levels  of  significance. 

Mathematically,  the  exact  level  of  significance  for  an  approxi¬ 
mate  chi-square  statistic  (S)  can  be  calculated  as 


(4-1) 


TO 


where  I  is  an  indicator  variable,  S(x^)  is  the  value  of  a  statistic 

given  a  contingency  table  realization  x^,  f (x^)  is  the  probability  of 

obtaining  the  realization  under  the  sampling  model  (e.g.,  multinomial), 

2 

and  Xjj  f  Is  the  chi-square  statistic  value  at  the  a  nominal  level 
and  appropriate  degrees  of  freedom  (d.f.).  The  sum  is  taken  over  all 
possible  table  realizations  given  the  sample  size  N  for  the  multi¬ 
nomial  sampling  model. 

It  should  be  apparent  that  these  exact  levels  will  generally  be 
different  than  the  nominal  levels.  The  chi-squared  distribution  is  con¬ 
tinuous,  so  that  any  a,  0  <  a  <  1,  could  be  chosen.  Yet,  for  a  given 
sample  size  there  are  only  a  discrete  number  of  possible  contingency 
table  arrangements,  so  only  a  discrete  number  of  possible  a^. 

ae  is  the  "exact"  probability  of  rejecting  a  true  hypothesis. 
Intuitively,  in  order  for  the  test  statistic  to  perform  well,  these 
exact  levels  should  be  close  to  the  nominal  levels  -  the  closer,  the 
better.  If  the  exact  level  is  less  than  the  nominal  level,  the  test  is 
conservative,  and  if  the  exact  level  is  greater  than  the  nominal  level, 
the  test  is  liberal.  A  conservative  test  is  usually  preferred  since  the 
nominal  level  provides  a  bound  on  the  probability  of  rejecting  a  true 
hypothesis.  Yet,  in  general,  8  increases  as  a  decreases,  so  that  a 
liberal  test  may  be  preferred  in  some  situations  to  protect  against  a 
type  II  error.  This  may  be  particularly  important  in  independence 
hypothesis  testing  situations,  where  the  purpose  often  is  to  find  vari¬ 
ables  that  are  independent  (i.e.,  accept  the  null  hypothesis). 

The  regression  example  in  Chapter  1  demonstrates  the  problems 
that  can  occur  when  the  chi-squared  distribution  does  not  well  approxi- 


mate  Che  exact  distribution  of  these  statistics.  The  exact  levels  of 


significance  provide  a  good  parameter  for  comparing  the  performance  of 
these  statistics  when  used  in  hypothesis  testing.  From  the  discussion 
above,  both  liberal  and  conservative  situations  need  to  be  considered. 
The  best  criterion  would  seem  to  be  how  close  these  exact  levels  are  to 
the  nominal  levels.  In  this  study  this  criterion  will  provide  the  basis 
for  the  comparison  of  these  statistics.' 

4.1.2  Zero  Cells  and  Zero  Marginals 

The  presence  of  cells  containing  zero  entries  and  of  marginals 
having  zero  values  can  cause  significant  problems  in  the  analysis  of 
contingency  tables  and,  in  particular,  in  the  use  of  the  chi-square 
statistics.  Zero  entries  in  contingency  tables  are  primarily  of  two 
types:  structural  or  sampling.  Structural  zeros  are  the  result  of 
sampling  variables  where  some  cross-classification  cannot  physically 
occur.  An  example  would  be  in  the  classification  of  male  and  female 
surgical  operations  where  there  could  be  no  cross-classified  cell  for 
male  hysterectomies.  Tables  of  this  nature  are  generally  defined  as 
"incomplete"  tables,  and  specialized  procedures  are  available  to  handle 
them  [e.g.,  see  Bishop  and  Fienberg  (1969),  Mantel  (1970),  or  Chen  and 
Fienberg  (1976)].  These  situations  will  not  be  investigated  in  this 
study.  Sampling  zeros,  on  the  other  hand,  are  the  result  of  the  fact 
that  there  is  some  probability  of  obtaining  an  observed  zero  entry  in 
a  table  of  finite  sample  size  under  a  given  sampling  model,  such  as 
the  multinomial.  But,  how  do  these  sampling  zeros  affect  the  analysis 

of  contingency  tables  using  the  methodologies  considered  in  this  study? 

2 

Both  the  Pearson  X  and  Kullback  21  statistics  depend  on  the 


2 

calculation  of  maximum  likelihood  estimates  (MLE) .  For  the  Pearson  X  , 

as  long  as  the  MLE  are  non-zero,  the  statistic  can  be  calculated  even 

in  the  presence  of  zero  observed  cells.  The  Kullback  statistic 

(21  *  2  £  x,  2n(x./m. ))  depends,  not  only  on  non-zero  MLE,  but  also  on 
i  1  11 

non-zero  cell  entries.  But,  noting  that  lim  n  In  n  ■  0,  the  contribu- 

n-w» 

tion  to  the  value  of  the  Kullback  statistic  for  cells  with  zero  observed 

entries  is  essentially  zero.  In  practice,  the  value  of  this  contribution 

is  set  at  zero,  or  as  suggested  by  Gokhale  and  Kullback  (1978b),  some 

small  value  (e.g.,  .000001)  is  given  to  the  zero  cell.  This  procedure 

seems  to  cause  very  little  bias. 

2 

The  GSK  statistic,  G  ,  is  the  most  affected  by  the  presence  of 
zero  observed  cells.  For  example,  consider  the  2*2  independence  test 
statistic, 

2  2 

G  -  (2n  x^  -  in  x12  -  in  x21  +  In  x22)  /£  £  (1/x^). 

When  any  observation  is  zero,  both  the  numerator  and  demoninator  become 

Infinite.  There  is  no  easy  way  to  separate  the  contribution  of  a  single 

2 

cell,  such  as  with  the  Pearson  X  and  Kullback  21,  and  show  a  limiting 
process.  Several  authors  have  suggested  ways  to  overcome  this  problem 
[e.g.,  see  Grizzle,  Starmer,  and  Koch  (1969),  Goodman  (1970),  Fienberg 
and  Holland  (1970),  and  Bhapkar  (1979)].  Grizzle  et.al.  (1969) 
recommend  adding  1/r  to  each  zero  cell,  where  r  is  the  number  of 
response  categories  (with  multinomial  sampling,  r  is  the  number  of 
cells).  This  "l/r  rule",  an  extension  of  a  rule  given  by  Berkson  (1955) 
for  binomial  sampling,  is  the  most  popular  rule  in  use  today  and  will 


be  used  in  this  study.  As  Forthofer  and  Lehnen  (1981,  p.  14)  suggest, 
"Replacement  of  the  zero  frequency  with  a  small,  non-zero  value  intro¬ 
duces  a  slight  but  acceptable  bias  into  the  estimate." 

A  more  significant  problem  in  the  use  of  these  statistics  occurs 
when  a  marginal  has  an  observed  zero  value.  With  respect  to  the  Pearson 
and  Kullback  statistics,  the  problem  is  easily  seen  since  these  statis¬ 
tics  depend  on  calculations  of  MLE.  As  shown  in  Chapter  III,  the  MLE 
depend  on  the  marginals  corresponding  to  the  hypothesis  of  concern.  If 
an  observed  marginal  is  zero,  meaning  that  all  the  corresponding  cells 
for  that  marginal  are  zero,  then  the  MLE  for  those  cells  will  be  zero 
since  these  marginals  must  be  preserved.  Neither  the  Pearson  nor 
Kullback  statistics  can  be  calculated  with  zero  MLE. 

Under  the  presense  of  zero  marginals  corresponding  -to  the  hypo¬ 
theses  of  concern,  a  more  fundamental  problem  occurs  affecting  all  three 
statistics.  This  problem  involves  the  very  nature  of  the  hypotheses  and 
is  best  described  through  the  log-linear  model. 

Consider  the  2  *  2  table  and  the  log-linear  model  reparameterized 
in  (3-15).  The  hypothesis  of  independence  corresponds  to  u^2  ■  0  (or 
of  Kullback' s  log-linear  model).  Assume  that  one  of  the  marginals  is 
equal  to  zero,  say  x#^  ■  0,  and  the  others  are  non-zero  as  reflected  in 
Figure  9. 


0  x 

•  2 

Figure  9.  2  *2  ’’’able:  One  Zero  Marginal  (x  ^  *  0) 


Under  the  null  independence  hypothesis  the  MLE  can  be  calculated  from 


(3-52), 


“ll  =  =  0 

®12  *  ^x2.)(x.2) 
™21  *  (x2»^ "  0 
®22  *  (x2.)(x.2} 


The  MLE  provide  estimates  for  the  probabilities. 


/N. 


These  estimates  for  p^  and  p2^  are  zero,  and  from  (3-19)  u^2  (or 
(3-82)T2)  cannot  be  estimated.  In  fact,  there  is  not  enough  information 
tc  perform  the  independence  test  under  any  criteria. 

Consider  now  the  2><3  table  with  one  of  the  three  category  mar¬ 
ginals  equal  to  zero,  say  x  ,  "0,  as  reflected  in  Figure  10. 


Figure  10.  2x3  Table:  One  Zero  Marginal  (x^  ■  0) 
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X12 

*13 
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X22 

x23 

0  x  ,  x  , 

•  2  *3 


The  corresponding  MLE,  m^  and  m^,  are  equal  to  zero.  In  the  2x3 
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log-linear  model  there  are  two  interaction  terms.  The  two  degrees  of 
freedom  independence  test  corresponds  to  both  terms  being  equal  to 
zero.  Depending  on  the  parameterization  used,  these  interaction  terms 
can  have  different  characterizations.  For  Kullback's  log-linear  model 
these  terms  are 


T3  =  £n  P11  “  P21  +  £n  p23  _  p13 

«  2n  p12  -  in  p22  +  in  p23  -  in  p13. 

Since  m^  *  m^  =  0,  there  is  no  estimate  for  t3>  and  the  two  degree  of 

freedom  test  cannot  be  performed.  However,  a  one  degree  of  freedom  test 

could  be  performed  based  only  on  T^.  In  effect,  this  would  be  collaps¬ 
ing  the  table  to  a  2x2  table,  disregarding  the  zero  cells  of  the  zero 
marginal.  The  t3  term  would  correspond  exactly  to  the  t3  term  of 
the  2x2  Kullback  log-linear  model.  The  only  effect  of  the  zero 

marginal  would  be  that  a  degree  of  freedom  is  lost. 

In  general,  this  problem  would  extend  to  larger  contingency  tables 

and  other  hypotheses.  A  zero  marginal,  where  the  marginal  relates  to 
the  hypothesis  of  concern,  will  cause  a  loss  in  the  degrees  of  freedom 
of  the  test  and,  in  effect,  collapse  the  table.  It  should  be  noted  that 
this  concept  of  collapsing  differs  from  the  usual  concept  of  collapsi- 
bility  of  contingency  tables  as  described  by  Bishop  (1971)  and  Bishop 
et.al.  (1975,  p.  47). 

Because  of  this  problem  associated  with  zero  marginals,  the  exact 


(or  estimated)  levels  of  significance  in  this  study  will  be  calculated 
conditionally  on  these  marginals,  associated  with  the  hypothesis  of 
concern,  not  being  equal  to  zero.  These  levels  of  significance  will 
then  be  conditional  probabilities.  From  the  discussion  above,  this  is 
Intuitively  desirable.  When  marginals  are  zero,  degrees  of  freedom  are 
lost,  and  the  hypothesis  test  is  made  with  the  appropriate  lower  degree 
of  freedom  chi-squared  distribution.  The  collapsed  contingency  table 
will,  in  effect,  be  an  observation  already  considered  with  respect  to 
the  calculation  of  exact  levels  of  significance  for  these  smaller 
tables. 

4.1.3  Probability  Designs 

The  exact  levels  of  significance  are  calculated  from  (4-1), 
based  not  only  on  the  null  hypothesis  and  the  statistic  but  also  the 
underlying  sampling  model.  For  the  tests  of  this  study  the  sampling 
model  is  multinomial  with  parameters  of  sample  size  and  cell  probabili¬ 
ties.  These  provide  the  natural  independent  parameters  for  the  experi¬ 
ment.  The  selection  of  sample  sizes  will  be  discussed  in  the  next 
section.  This  section  will  discuss  the  selection  of  the  cell  probabili¬ 
ties,  which  in  vector  form  will  be  probability  vectors. 

In  selecting  these  probability  vectors,  two  considerations  are 
important.  First,  the  design  must  be  large  enough  in  teems  of  the  range 
of  probabilities  to  encompass  the  expected  range  of  variability  of  the 
exact  levels  of  significance  and  provide  trend  information.  Second,  the 
design  must  be  small  enough  in  terms  of  number  of  vectors  selected  to  be 


manageable  within  the  constraint  of  available  computer  time. 


The  null  hypotheses  of  independence  conveniently  provide  the 
basis  for  the  selection  of  the  underlying  probability  vectors.  For 
example,  the  null  hypothesis  of  independence  for  two-way  tables  is 

V  »«  ■  Oi.p.r  1-M . .  j-1>2 . s- 

By  specifying  the  marginal  probabilities  (p^,p^),  under  the  null 

hypothesis  the  complete  probability  structure  is  determined.  A  similar 

situation  exists  for  three-way  tables  and  the  complete  independence 

hypothesis,  where  the  one-way  marginals  (p.  ,p  ,  ,p  .)  are  sufficient. 

i* •  *j  •  »k 

Table  5  provides  a  partitioning  of  the  degrees  of  freedom  associ¬ 
ated  with  each  table  and  hypothesis  test  used  in  this  study.  Included 
in  the  table  is  a  list  of  those  marginals  to  be  specified.  Note  that 
not  all  marginal  probabilities  need  be  specified  since  the  multinomial 
constraint,  that  the  probabilities  sum  to  one,  will  uniquely  determine 
one  marginal  if  the  other  corresponding  marginals  are  given.  In  Chapter 
III  it  was  shown  that  the  hypothesis,  Hq:  p^  *  pl.p.l»  ^or  t*ie  2  *  2 
table  was  sufficient  for  the  one  degree  of  freedom  hypothesis  test  for 
independence  (which  requires  that  p^  -  pi.p*j  ^or  Similar 

results  hold  for  other  tables  and  Independence  tests,  and  the  required 
hypothesis  tests  for  this  study  are  listed  in  Table  5.  Other  combina¬ 
tions  of  tests  could  be  chosen  that  would  also  suffice.  The  ones 
listed  in  Table  5  correspond  to  the  lowest  lexicographic  order. 

Except  for  the  no  second-order  interaction  test  in  the  2x  2x  2 
table,  the  tests  and  required  marginals  in  Table  5  are  fairly  straight¬ 
forward.  For  the  no  second-order  interaction  test  a  simple  combination 


Table  5.  Hypothesis  Tests  and  Specified  Marginals 


Degrees 

of 

Freedom* 


Hypothesis  Tests  and  Specified  Marginals 


Independence  H^:  p^  *  p  p^ 


Specify  p1#,  p#1 


2  Independence  HQ:  pn  -  P12  “  pl.p-2 


Specify  p1#,  P#1,  P>2 


Independence  HQ:  pu  -  p^p^,  p12  -  P1#P.2 
P13  Vl.P.3 

Specify  p1#,  P>1,  p.2,  P>3 


Independence  HQ:  Pu  ■  P^P.2*  pi2  "  pi.p.2 


P21  "  P2*P*1*  p22  “  P2-P*2 


Specify  p1#,  p2<>  p.1#  p.2 


Independence  HQ:  pu  -  P12  "  p 


p13  *  pl.p.3’  p14  "  p: 


Specify  p^4,  p^,  p<2,  p<3>  p^ 


*Each  table  has  1  degree  of  freedom  associated  with  the  normal  con¬ 
straint  Ep  »  1. 
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Degrees 

of 

e  Freedom* 


2x2x2 


2x2x2 


2x2x2 


Table  5.  Continued 


Hypothesis  Tests  and  Specified  Marginals 


Complete 

Independence 


Pl*  *P*1*P* •!' 


pl..p.l.p..2» 


pl. .p.2.p. *1’ 


p2» .p.]_.p. 


Specify  p1#>>  ptl>,  p.#1 


Complete 


Independence 


pl* .P-i-P. .1* 
pl..  p.l.p..  2’ 


pl..  p.  l.  p. .  3* 
pl..p.  2.  p..l» 
pl..p.  2*P**2’ 


P2*  •  P«  l»  p»  •  l* 


p2» . p.  l. p. .  2 


Specify  p1>#,  P>1.,  p.^,  p..2 


Interaction"^"  V  *^*122*212*  221 

p112p121p211p222 


6  Specify  pu.,  P12..  P^.  P2.r  P.u.  P.12 

*Each  table  has  1  degree  of  freedom  associated  with  the  normal  con¬ 
straint  Ip  ■  1. 


,> 
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of  one-way  marginals  will  not  suffice.  The  hypothesis  has  only  one 
degree  of  freedom,  leaving  six  degrees  of  freedom  for  the  specifica¬ 
tion  of  parameters  to  determine  uniquely  the  under  the  null 

hypothesis.  The  combination  of  given  two-way  marginals  will  suffice, 
as  well  as  any  other  independent  set.  With  these  two-way  marginals  an 


iterative  procedure,  similar  to  that  given  for  the  MLE  in  Chapter  III, 


could  be  used  to  determine  uniquely  the  cell  probabilities.  An  easier 


combination  of  six  independent  parameters  would  be  any  six  p^^*  The 


other  two  P^j^  could  then  be  computed  from  the  null  hypothesis  and  the 


multinomial  constraint,  ][  p. »  1. 

i.j,k 


Once  the  probability  parameters  have  been  designated,  the 


question  still  remains  of  how  these  parameters  should  be  varied  in  order 


to  insure  a  sufficient  range  for  the  probability  structure.  After  some 


preliminary  investigation  and  consideration  of  the  need  to  have  some 


structures  with  very  small  expected  values,  it  was  decided  to  allow  the 


marginals  to  have  values  as  small  as  .1.  In  effect,  this  would  provide 


some  cells  with  probabilities  as  small  as  .01  for  the  two-way  tables 


and  .001  for  the  three-way  tables.  Using  sample  sizes  under  100  this 


would  guarantee  some  tables  with  very  small  expected  values. 


Considering  first  the  2x2  table,  by  allowing  the  marginals 


(p^,p  to  range  from  .1  to  .9  in  increments  of  .1  and  taking  the  com¬ 


plete  convolution,  there  would  be  81  vectors.  However,  these  vectors 


are  not  unique.  It  makes  no  difference  which  variable  is  variable  one 


and  which  variable  is  variable  two.  The  interaction  term  of  the  log- 


linear  model,  u^2>  will  be  unaffected;  so  that,  letting  p^a  ■  .1  and 
p(^  *  .2  is  equivalent  to  letting  p^(  *  .2  and  pa^  *  This  reduces 
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the  number  of  vectors  to  45. 

Additionally,  there  Is  a  symmetry  with  respect  to  the  categories 
of  each  variable.  A  simple  proof  will  demonstrate  this  symmetry.  Let 
p^  and  p^  be  selected,  then  under  Hq 


pl.p.l 


l.«2 


p2-p.l 


2»  »2* 


where  the  probabilities  can  be  represented  in  the  standard  arrangement 
of  Figure  11. 


P11  P12 


P21  p22  P2. 

P-1  p.2  1 


Figure  11.  2x2  Table:  Standard  Arrangement 


Now,  let  another  selection  of  the  marginals  be  made  where  pj^  "  p^a 

and  p',  ■  1  -  p  ,  *p  ,,  then  under  H_ 

►l  • i  »z  o 


ms 


145 


pil  "  pl.p!l  "  pl.p.2  "  p12 
p12  "  pi.p’.2  “  pl.p.l  "  P11 
P21  *  PLPll  "  p2.p.2  "  P22 

p22  "  P2»P«2  "  p2.p.l  "  P21 

with  the  arrangement  of  original  probabilities  reflected  in  Figure  12. 


p2. 

1 

Figure  12.  2*2  Table:  Interchanged  Categories 

In  effect,  the  categories  of  variable  two  have  been  interchanged.  With 
respect  to  the  log-linear  model  this  change  will  simply  change  the  sign 
of  the  interaction  term.  The  chi-square  tests  for  independence  will  be 
unchanged. 

These  symmetry  arguments  have  now  reduced  the  81  vectors  to  only 
15.  The  marginals,  p^a  and  p^,  range  from  .1  to  .5  by  increments  of  .1 
and  are  convoluted  to  form  the  complete  design  (given  in  Appendix  B)  for 
the  2x2  table. 

Symmetry  arguments  for  the  2  x  3  table  are  similar  with  respect 


P12 

P11 

p22 

P21 

to  change  of  categories,  but  the  variables  cannot  be  interchanged  since 
they  each  have  a  different  number  of  categories.  However,  results  for 
the  3x2  table  would  be  equivalent  to  those  for  the  2x3  table.  From 


Table  5  there  is  a  requirement  to  specify  the  three  marginals  (p1  .p^, 
p^)*  Letting  p^  *  .1  and  range  to  .5,  p^  -  .1  and  range  to  .3,  and 
p^2  "  P.j^  and  range  to  .4,  all  in  increments  of  .1,  the  entire  convolu¬ 
tion  will  Include  40  vectors.  From  a  preliminary  investigation  it  was 
thought  that  it  would  be  too  costly  in  terms  of  computer  time  to  use 
the  entire  design,  and  that  similar  information  could  probably  be  ob¬ 
tained  using  only  one-half  the  number  df  vectors.  It  was  decided  to 
use  every  other  vector  in  the  lexicographic  ordered  design  of  varying 
marginals.  In  addition,  the  equiprobable  vector  with  marginals  -  1, 
and  p#^  ■  p^2  “  1/3,  giving  equiprobable  cell  probabilities  p^  •  1/6 
for  all  1  and  j,  was  included.  The  final  <V  sign  of  21  vectors  is  given 
in  Appendix  B. 

The  2^4  and  2^5  tables  had  similar  complete  designs  of  40  and 
32  vectors,  respectively.  Again,  every  other  vector  was  used,  and  the 
equiprobable  case  added  to  the  2x4  table,  to  give  designs  with  21  and 
16  vectors.  The  3x3  table,  like  the  2x2  table,  has  the  additional 
symmetry  property  of  interchange  of  variables.  A  total  of  36  vectors 
were  available  in  the  complete  design.  Including  the  equiprobable  case 
21  vectors  were  used  for  the  study.  All  these  designs  are  listed  in 
Appendix  B. 

The  2x2x2  table  under  the  hypothesis  of  complete  Independence, 
like  the  2x2  and  2x3  tables ,  has  the  symmetry  property  of  interchange 
of  any  two  variables,  as  well  as  the  interchange  of  categories,  without 
affecting  the  chi-square  test  statistics.  The  complete  design  includes 
34  vectors.  Under  a  similar  scheme  of  roughly  including  every  other 
vector  and  the  equiprobable  vector,  22  vectors  were  used  for  the  design 
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The  2  x  2  x  2  table  under  the  hypothesis  of  no  second-order  inter¬ 
action  presents  some  specialized  problems.  First,  as  reflected  in  Table 
5,  under  this  hypothesis  there  is  no  convenient  set  of  marginals  that 
will  easily  generate  the  cell  probabilities.  If  marginal  probabilities, 
such  as  those  in  Table  5,  are  used,  an  iterative  scheme  would  be  needed 
to  find  the  cell  probabilities.  Any  six  cell  probabilities  could  be 
specified  to  satisfy  the  six  degrees  of  freedom,  but  any  attempt  to 
cover  the  range  of  possibilities  for  these  specified  probabilities 
would  seem  fruitless.  As  noted  in  Chapter  III,  the  no  second-order 
interaction  hypothesis  is  a  sub-hypothesis  of  the  complete  Interaction 
hypothesis;  so  that,  the  probability  vectors  selected  under  complete 
interaction  also  satisfy  the  hypothesis  of  no  second-order  interaction. 
Therefore,  it  was  decided  to  use  this  same  design  for  no  second-order 
interaction  hypothesis  in  the  2x2x2  table.  Unfortunately,  another 
problem  occurred.  As  discussed  in  Section  4.1.2,  zero  observed  margi¬ 
nals  corresponding  to  the  hypothesis  of  concern  will  invalidate  the 
test  at  the  given  degrees  of  freedom.  The  twelve  two-way  marginals  of 
the  2x2x2  table  correspond  to  the  no  second-order  interaction  hypo¬ 
thesis.  In  the  generation  of  random  tables,  restricting  these  marginals 
to  be  non-zero  for  the  extreme  vectors  in  the  2x2x2  complete  inde¬ 
pendence  design  causes  many  observations  to  be  discarded.  A  preliminary 
study,  using  the  most  extreme  probability  vector,  required  about  800 
tables  to  be  generated  in  order  to  find  five  that  had  no  two-way 
marginals  equal  to  zero.  It  was  decided  to  discard  the  five  most 
extreme  vectors,  leaving  the  design  of  17  vectors  listed  in  Appendix  B. 


For  the  2x2x3  table,  under  the  complete  independence  hypothesis 
the  number  of  vectors  for  the  complete  design  is  increased  to  120.  This 
is  the  result  of  the  fact  that  the  third  variable  cannot  be  interchanged 


with  the  other  two;  hence,  a  complete  convolution  with  respect  to 
this  variable  is  required.  To  reduce  the  number  of  vectors  used  and 
still  cover  the  range  of  possible  vectors,  it  was  decided  to  use  every 
fourth  vector  and  include  the  equiprobable  vector  to  give  a  design  of 
31  vectors  as  listed  in  Appendix  B. 

This  completes  the  designs  of  the  underlying  probability  struc¬ 
tures  for  the  various  tables  and  hypothesis  tests.  It  is  believed  that 
these  reflect  the  range  of  what  would  occur  in  actual  hypothesis  testing 
situations  and  should  give  'indications  of  trends  with  respect  to  any 
parameters  that  may  be  considered. 

4.1.4  Sample  Sizes 

While  specific  sample  sizes  may  not  be  as  important  as  the  under¬ 
lying  probability  structure,  care  should  be  taken  to  allow  consideration 
of  those  sample  sizes,  given  the  probability  structures,  that  will 
allow  cell  expected  values  to  range  from  relatively  small  to  relatively 
large.  The  relationship  between  sample  size  (N)  and  expected  value  (m) 
is  m  ■  Np,  where  p  is  the  probability  of  the  corresponding  cell. 

It  might  be  expected  that  sample  sizes  can  be  as  small  as  desired 
Unfortunately,  the  zero-marginal  constraint,  previously  discussed,  rea¬ 
sonably  requires  that  some  lower  bound  be  placed  on  the  sample  size. 

The  one  exception  to  this  is  the  2x2  table  where  an  "exact"  procedure 
and  program  were  developed,  making  it  very  easy  to  calculate  exact 
levels  of  significance  for  any  small  sample  size  (N  a  multiple  of  four). 


The  other  tables  use  a  Monte  Carlo  procedure  where  small  sample 
sizes  require  many  more  tables  to  be  generated  before  a  sufficient 
number  of  "usable"  (no  zero  marginals  corresponding  to  the  hypo¬ 
thesis  of  concern)  tables  are  available. 

For  a  maximum  sample  size  value,  somewhere  near  100  was  thought 
to  be  sufficient.  This  would  give  expected  values  for  the  equiprobable 
tables  from  10  for  the  2x5  table  to  25  for  the  2x2  table  and  from 
8.33  for  the  2x2x3  table  to  12.5  for  the  2x2x2  table.  Based  on  a 
preliminary  investigation,  100  also  seemed  reasonable  with  respect  to 
computer  generation  times. 

Having  chosen  the  range  of  values,  it  was  important  to  consider 
what  specific  sample  sizes  would  be  used  for  each  design.  It  was 
thought  that,  since  comparisons  were  to  be  made  across  tables,  some 
criteria  to  standardize  these  sample  sizes  would  be  useful.  Consider 
the  parameter,  average  expected  value,  m  *  N/k,  where  the  k  is  the  number 
of  cells  (k*rs  for  a  two-way  table).  Roscoe  and  Byars  (1971)  based  their 
recommendations  on  this  quantity.  Usually,  the  magnitudes  of  the  expect¬ 
ed  values  (based  on  estimates)  are  unknown  until  after  a  sample  is  taken. 
Because  of  this,  Roscoe  and  Byars  felt  that  their  results  would  be  most 
valuable  reported  for  their  uniform  (equiprobable)  case.  Also,  they 
state  that,  "there  is  an  appealing  simplicity  to  this  approach." 

What  seems  even  more  Important  for  the  study  of  this  thesis  is 
the  need  for  comparisons  across  tables.  With  this  in  mind  sample  sizes 
were  chosen  so  that  the  average  expected  values  would  be  integer.  The 
general  comparable  range  across  tables  for  these  average  expected  values 
are  from  two  to  12,  although  the  2x2  and  2x3  tables,  in  particular, 
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Include  higher  values.  Also,  during  the  testing  of  each  Monte  Carlo 
program,  some  adjustments  of  sample  sizes  were  made  to  control  computer 
times.  For  example,  the  minimum  N  for  the  2x2x2  table  under  the  no 
second-order  interaction  hypothesis  was  changed  from  16  to  24  due  to 
previously  discussed  zero-marginal  problems. 

The  complete  sample  size  design  is  given  in  Appendix  C.  These 
sample  sizes  were  used  with  each  probability  vector  of  the  probability 
designs  in  Appendix  B. 


4.2  The  Exact  2x2  Program 

The  primary  purpose  of  developing  an  exact  program  for  the  2x2 
table  was  to  provide  data  for  validating  the  Monte  Carlo  simulation. 

The  exact  data,  along  with  Monte  Carlo  data  for  large  sample  sizes,  were 
also  used  in  the  basic  analysis  of  the  2x2  table.  A  listing  of  the 
exact  program  is  given  in  Appendix  D>and  the  exact  data  generated  for 
the  complete  2x2  design  are  given  in  Appendix  F. 

The  procedure  used  in  the  program  is  based  on  a  partial  enumera¬ 
tion  of  all  possible  tables  and  on  symmetry  properties.  The  key  to  the 
procedure  is  restricting  the  sample  size  N  to  be  divisible  by  four. 

This  allows  for  significant  computational  efficiencies  to  be  achieved. 

Tables  were  first  classified  based  on  their  symmetry  properties 
with  respect  to  the  three  chi-square  statistics.  Formulas  were  then 
derived  which  provided  information  on  the  number  of  tables  to  be 
generated  for  a  given  sample  size.  Then,  procedures  were  developed  to 
enumerate  the  tables  within  each  classification. 

The  first  formula  derived  was  for  the  total  number  of  "usable" 
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tables  for  a  given  sample  size.  This  formula  for  any  N  >  1  is 


(N3  +  6N2  -  3N  +  6)/6. 


However,  many  of  these  tables  are  not  unique  in  the  sense  that,  due  to 
symmetry  properties,  a  chi-square  statistic  will  have  the  same  value 
for  several  tables.  Formulas  for  the  dumber  of  "unique"  (with  respect 
to  the  three  statistics),  "usable"  tables  were  derived  for  various 
sample  sizes.  These  are 


(N3  +  9N2  -  N  -  9)/48  for  N  odd, 

(N3  +  9N2  +  8N)/48  for  N  even  and  divisible  by  4, 

(N3  +  9N2  +  8N  -  12) /48  for  N  even  and  not  divisible  by  4. 


Table  6  gives  a  comparison  of  the  total  and  unique  number  of  tables  for 
various  sample  sizes.  From  the  formulas  it  is  easy  to  see  that  the  ratio 


Table  6.  2x2  Total  and  Unique  Number  of  Tables 


N 

Total 

Unique 

N 

Total 

Unique 

N 

Total 

Unique 

4 

19 

5 

12 

407 

65 

32 

6417 

880 

5 

36 

7 

14 

624 

96 

36 

8995 

1221 

6 

60 

12 

16 

905 

136 

40 

12181 

1640 

7 

92 

16 

18 

1258 

185 

44 

16039 

2145 

8 

133 

24 

20 

1691 

245 

48 

20633 

2744 

9 

184 

30 

24 

2829 

400 

52 

26027 

3445 

10 

246 

41 

28 

609 

56 

32285 

4256 

of  total  to  unique  tables  is  approaching  1/8.  Using  the  symmetry 
properties,  at  N  a  56  only  13  percent  of  the  total  number  of  tables 
need  to  be  generated  -  a  considerable  savings. 

Considering  the  symmetry  properties,  the  tables  were  classified 
into  five  groups  (E^,  E3»  0) •  Table  7  presents  this  classifi¬ 

cation  scheme  for  N  divisible  by  four  along  with  formulas  for  the 
number  of  unique  tables,  formulas  for  the  number  of  total  tables  within 
each  classification,  and  a  specification  of  the  number  of  arrangements 
for  each  unique  table  within  each  classification  group.  The  primary 
purpose  for  this  classification  scheme  relates  to  how  many  of  the 
total  number  of  tables  are  equivalent  to  a  given  "unique"  table;  i.e., 
how  many  symmetric  arrangements  are  possible  for  each  unique  table. 

For  example,  group  defines  those  tables  with  all  cells  equal,  but 
for  a  given  sample  size  (N)  divisible  by  four,  there  is  only  one  such 
table.  Group  E2  defines  those  tables  where  both  sets  of  corners  are 
equal  (in  lexicographic  order  this  is  (abab)  with  a#>).  There  are  N/4 
such  "unique"  tables,  and  for  each  "unique"  table  there  are  two 
arrangements  included  in  the  total  number  of  tables;  i.e.,  (abab)  and 
(baba).  Note  that  the  total  number  of  tables  can  be  found  by  multiply¬ 
ing  the  number  of  unique  tables  by  the  number  of  arrangements,  and  that 
the  grand  totals  correspond  to  the  formulas  previously  given. 

The  exact  levels  of  significance  are  essentially  calculated  by 
summing  multinomial  probabilities  for  tables  where  the  particular 
approximate  chi-square  statistic  is  greater  than  the  value  of  the  chi- 
square  statistic  (with  one  degree  of  freedom)  at  a  specified  nominal 
level.  The  procedure  begins  by  generating  a  table,  {x^},  within  a 


particular  symmetric  classification.  The  total  multinomial  probability 


f(x  )  -  I  f(x  )  -  l 

XiJ£Xg  Xij£Xg 


N! 


p«J/v 


is  calculated  for  each  unique  table  group  (xg)  by  using  the  generated 
table  and  rotating  it  through  its  symmetric  arrangements.  The  three 
statistics,  Sig(i-1,2,3) ,  are  then  calculated  for  this  particular 
unique  table  group.  Hypothesis  tests  are  then  performed  at  each 


nominal  level  of  significance  (a  -  .10,  .05,  .01)  by  comparing  the 

2 

statistics  with  the  ±  statistics.  The  multinomial  probability 
associated  with  a  rejected  hypothesis  is  accumulated. 


I  «v  •  *ig* 


all  g 


.a 


where  Iig  is  an  indicator  variable  for  the  hypothesis  test;  i.e.. 


E? 

ig 


x*  lf.si«  =•  Vi 

0,  otherwise  . 


Also,  the  total  used  probability  is  accumulated. 


F 


l 

all  g 


f(xg). 


The  sums  are  taken  over  all  unique  table  groups.  It  should  be  noted 
that  F  will  not  equal  one  since  only  "usable"  tables  are  considered. 


Exact  significance  levels  for  each  statistic  (S^)  and  each  nominal  a  are 
then  calculated  as  the  ratio  of  these  two  accumulations, 


l 

all  g 


(f(xg) 


)/F. 


It  is  clear  that  these  exact  significance  levels  are  conditional  pro¬ 
babilities. 

As  a  check  that  all  required  tables  were  considered,  an  exact 
formula  for  F  was  derived.  Letting  Z  be  the  probability  of  a  zero 
marginal, 


_  N  ,  N  .  N  .  N  v  N 

Z  ’  Pl-  +  P2.  +  "-I  +  p.2  '  l.  ■’ll  • 


F  is  the  probability  of  a  "usable”  table,  so 


F-  l  -  Z. 


F  values  were  calculated  for  each  design  point  and  compared  to  the 
accumulated  totals  in  the  program.  Agreement  was  exact  to  the  four 
decimal  places  of  the  output  data. 

4,3  The  Simulation  Program 

The  Monte  Carlo  simulation  program  is  the  heart  of  this  study. 


It  provides  most  of  the  data  for  the  analysis  and  the  results  of 
Chapter  V.  The  purpose  of  the  simulation  is  to  provide  estimates  of 
exact  levels  of  significance  for  the  competing  statistics  at  designated 


nominal  levels  and  under  the  probability  and  sample  size  designs  pre¬ 
sented  in  Section  4.1.  This  section  will  discuss  the  design  of  the 
Monte  Carlo  program  and  its  validation  and  the  selection  of  the  number 
of  tables  to  generate  for  each  design  point. 

4.3.1  Program  Design 

A  separate  computer  program  was  written  for  each  type  table 
(2*2,  2x3,  ...,  2x2x3)  and  hypothesis  test  (independence,  complete 
independence,  no  second-order  interaction).  The  efficiency  of  the  pro¬ 
gram  was  paramount,  and  by  using  separate  programs,  maximum  advantage 
could  be  taken  of  the  unique  structure  of  each  table  and  hypothesis 
test.  Appendix  G  provides  a  listing  of  the  program  for  the  2x3  table. 
Each  of  the  other  programs  follows  the  same  general  procedure  and  formats. 

"Each  program  is  divided  into  four  parts:  a  main  program  and 
three  subroutines.  The  main  program  controls  the  cycling  process 
through  the  various  probability  designs  and  sample  sizes,  calls  the 
three  subroutines,  makes  the  final  calculations  for  the  estimated  exact 
levels  of  significance,  and  prints  the  results.  The  three  subroutines 
will  be  discussed  below  in  the  order  that  they  are  used. 

Subroutine  GEN  is  the  most  important  part  of  the  simulation. 

The  subroutine  uses  the  IMSL  (1980)  subroutine  GGMTN  coding,  modified, 
for  efficiency,  to  the  specific  problem  of  generating  random  multi¬ 
nomial  observations  of  size  k  (k  equal  to  the  number  of  cells)  with 
parameters  given  by  the  probability/sample  size  designs.  The  proce¬ 
dure  begins  by  generating  a  binomial  random  observation  based  on  using 
the  multinomial  probability  (p^)  for  a  cell  and  the  sample  size,  N. 


The  binomial  observation  will  be  this  cell's  multinomial  observation 


(x^).  To  generate  the  observation  for  the  next  cell,  its  multinomial 
probability  is  first  adjusted  to  a  conditional  probability  (given 
that  the  first  cell  has  been  observed)  by  dividing  by  the  unused  pro¬ 
bability  (p£  "  •  Th«  sample  size  is  also  adjusted  by  sub¬ 

tracting  the  random  observation  of  the  first  cell  (N'  ■  N-x^) .  These 
parameters  (N',  p’)  are  then  used  to  generate  another  random  binomial 
observation,  which  is  the  multinomial  Observation  (x_)  for  the  second 
cell.  The  procedure  is  continued  until  one  cell  remains.  The  obser¬ 
vation  for  this  cell  is,  with  conditional  probability  equal  to  one, 
equal  to  the  unused  sample  size  (N-x^-x^. . .-x^_^) . 

The  procedure  above  Includes  the  use  of  the  IMSL  binomial  random 

variate  generator,  subroutine  GGBN.  This  subroutine  uses  a  basic 

counting  routine  for  N  less  than  35,  where  uniform  (0,1)  random  numbers 

2  3 

are  generated  and  compared  to  the  binomial  probability  p,  then  p  ,  p  , 
4 

and  p  .  This  procedure  allows  for  up  to  four  realizations  of  the 
binomial  random  variate  with  only  a  generation  of  one  uniform  random 
number.  For  N  greater  than  34  GGBN  uses  a  method  reported  by  Relies 
(1972)  based  on  the  use  of  medians  from  random  samples  of  a  uniform 
distribution.  The  medians  are  "efficiently"  generated  as  beta  random 
variates . 

The  procedure  above  for  N  less  than  35  is  exact,  in  the  sense 
that  the  "randomness"  of  the  observations  depends  only  on  the  random 
number  generator  itself.  Relies'  procedure  for  N  greater  than  34  is 
approximate.  Relies  derives  upper  bounds  on  the  probability  of  error 
in  using  his  procedure  for  various  "threshold"  values  of  N.  In  the 
GGBN  subroutine  the  threshold  value  is  34.  For  a  threshold  value  of 
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31  Relies  reports  error  probabilities  of  .00030,  .00042,  and  .00046 

for  sample  sizes  of  32,  64,  and  128,  respectively.  For  the  study  of 

this  thesis  a  reasonable  bound  for  the  probability  of  an  error  in  any 

one  cell  would  then  be  .0005.  An  error,  once  made,  would  propagate 

through  the  other  cells,  but  the  probability  of  making  two  errors  in 

2 

one  multinomial  observation  would  be  bounded  by  (.0005)  .  This  bound 

for  the  probability  of  multiple  errors  would  continue  to  multiply  until 

the  remaining  N*  was  less  than  35  and  the  "exact"  counting  method  used. 

It  is  clear  that  this  procedure  will  be  quite  accurate. 

The  IMSL  routines  for  random  number  generation  used  in  the  GGBN 

subroutine  are  GGUBS  and  GGUFBS,  where  GGUFBS  is  the  function  form  of 

GGUBS.  These  routines  use  a  linear  congruentlal  generator  first  reported 

by  Hutchinson  (1966) .  The  particular  version  with  multiplier  16807  and- 
31 

modulus  2  >1  was  first  presented  and  tested  by  Lewis,  Goodman,  and 
Miller  (1969)  for  the  IBM  360  computer.  The  generator  has  been  exten¬ 
sively  tested  by  Learmonth  and  Lewis  (1973)  and  shown  to  have  excellent 
statistical  properties. 

As  part  of  subroutine  GEN,  after  a  multinomial  random  observation 
is  generated,  the  corresponding  contingency  table  is  checked  for  appro¬ 
priate  zero  marginals.  If  a  marginal  is  zero,  the  observation  is 
discarded  and  a  new  observation  is  generated.  The  process  continues 
until  an  appropriate  number  of  "usable"  random  tables  have  been  generated. 

Once  these  tables  have  been  generated,  subroutine  STATS  is  called. 
This  subroutine  calculates  maximum  likelihood  estimates  and  the  three 
contingency  table  statistics  for  each  table.  The  calculations  for  the 
Pearson  and  Kullback  statistics  are  fairly  straightforward.  Each  cell 


provides  a  contribution  to  the  value  of  the  statistic,  h  zero  cells 
providing  a  contribution  of  zero  to  the  Kullback  statistic  (see  Section 
4.1.2).  The  calculation  of  the  GSK  statistic  is  more  involved.  As 
presented  in  Chapter  III,  this  statistic  requires  an  inversion  of  the 
asymptotic  covariance  matrix.  Conveniently,  for  the  2xs  tables  this 
matrix  is  highly  structured  with  all  off-diagonal  terms  equal,  and  the 
inversion  can  be  derived  in  a  relatively  simple  closed-form.  For  the 
2*2*2  table  under  the  complete  independence  null  hypothesis ,  the  four 
degree  of  freedom  GSK  test  statistic  requires  the  inversion  of  a  4  *  4 
matrix  that  does  not  have  the  convenient  structure  of  the  2xs  tables. 
An  efficient  IMSL  symmetric  matrix  inversion  subroutine,  L1NV1P,  was 
used.  As  will  be  discussed  in  Chapter  V,  the  GSK  statistic  was  not 
calculated  for  the  2*5,  3*3  and  2*2*3  tables. 

Once  the  statistics  for  each  table  are  calculated,  subroutine 
HYP  is  called.  This  subroutine  performs  a  hypothesis  test  on  each  of 
the  tables  for  each  statistic  at  each  nominal  level  of  significance 
(.10,  .05,  .01),  based  on  the  value  of  the  chi-square  statistic 
specified  to  six  significant  digits  [Pearson  and  Hartley  (1954,  Vol.  I, 
Table  8,  pp.  130-134)].  The  subroutine  accumulates  the  number  of  times 
the  hypothesis  test  rejects  for  each  statistic  at  each  nominal  level. 

Once  all  of  the  tables  have  been  tested,  the  rejection  totals 
are  returned  to  the  main  program.  The  main  program  calculates  the 
estimated  "exact"  levels  of  significance  by  dividing  the  rejection 
totals  by  the  total  number  of  tables  generated.  These  estimates  are 
then  printed  for  each  probability/sample  size  design,  statistic,  and 


nominal  level 
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4,3.2  Number  of  Tables 

A  critical  aspect  In  the  design  of  the  Monte  Carlo  procedure 
was  the  determination  of  the  number  of  tables  to  generate  for  each 
estimate  of  the  exact  levels  of  significance.  The  previous  studies, 
reviewed  In  Chapter  II,  used  1000,  2000,  5000,  and  10000  tables.  With 
the  size  of  the  designs  to  be  considered  in  this  study,  it  seemed 
crucial  to  limit  the  number  of  tables  to  be  as  low  as  possible  and 
still  give  statistically  reliable  data.  It  was  decided  to  consider 
accuracy  levels,  corresponding  to  confidence  intervals  about  the  nomi¬ 
nal  levels  of  significance,  for  various  numbers  of  tables.  These  levels 
depend  on  the  normal  approximation  to  the  binomial  distribution. 

From  the  procedure  described  in  the  previous  section,  let  X  be 
the  number  of  "rejections"  in  K  tables.  Then  X  has  a  binomial  distri¬ 
bution  with  probability  p,  where  p  is  the  "exact"  probability  of  rejec- 

2 

tlon,  and  the  mean  and  variance  are  y  ■  Kp  and  a  -  Kp(l-p) .  Then 
1/2 

(X-Kp)/ (Kp(l-p))  has  an  approximate  normal  (0,1)  distribution  for 

large  K.  Letting  E  be  the  maximum  "error"  (accuracy)  for  the  random  vari- 

1/2 

able  X,  with  probability  1-a,  E  *  Z^^CpCl-p)^)  .  For  the  nominal 
values  of  p(. 10, .05, .01)  to  be  considered  in  this  study,  errors  were 
calculated  with  probability  .95  (corresponding  to  95  percent  confidence 
intervals)  for  various  numbers  of  tables  (K)  and  are  shown  in  Table  8. 

Besides  these  accuracy  levels,  preliminary  computer  program  tests 
were  performed  to  estimate  computer  times  and  to  observe  the  effects 
of  increasing  K  by  using  a  moving  average  of  estimated  levels.  Based 
on  these  tests,  the  previous  studies,  and  the  estimated  .95  accuracy 
levels  of  Table  8,  2000  tables  were  selected.  A  significant  factor  in 


Table  8.  .95  Accuracy  Levels  (E) 


Nominal  p 

K 

E 

.10 

1000 

.0186 

2000 

.0131 

5000 

.0083 

10000 

.0059 

.05 

1000 

.0135 

2000 

.0096 

5000 

.0060 

10000 

.0043 

.01 

1000 

.0062 

2000 

.0044 

5000 

.0028 

10000 

.0020 

K  ■  No.  Tables 


this  selection  was  the  .95  accuracy  level  at  the  nominal  value  of  .05. 
Table  8  shows  this  level  to  be  .0096,  which  would  correspond  to  a  95 
percent  confidence  interval  approximately  (.04,  ,06).  Several  authors 
beginning  with  Cochran  (1952),  have  used  this  interval  as  a  measure  of 
performance  of  the  chi-square  statistics  at  the  .05  nominal  level. 
4.3.3  Program  Validation 

In  order  for  any  simulation  model  to  be  useful  as  an  investiga¬ 
tive  tool,  it  must  be  reliable  and  consistent;  that  is,  the  simulation 
should  provide  reasonable  results.  To  check  these  properties,  it  is 
often  desirable  to  provide  a  method  of  validation.  Validation  is 
usually  accomplished  by  checking  the  output  of  the  simulation  for  some 
of  the  values  of  the  input  parameters  where  the  true  output  is  known. 
This  is  especially  important  with  discrete  event  simulation.  In 
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Monte  Carlo  simulation  this  is  often  not  essential  since  exact 
formulas  are  usually  used  for  the  generation  procedure,  and  empirical 
results  or  tests  are  available  to  provide  sufficient  information  on 
the  simulation  validity.  This  is  the  case  for  the  Monte  Carlo  simu¬ 
lation  of  the  study  in  this  thesis.  Errors  could  occur  as  a  result 
of  problems  with  the  random  number  generator  or  as  a  result  of  the 
approximate  binomial  generation  scheme  for  N  greater  than  34.  As 
previously  discussed,  the  tests  and  error  bounds  indicate  that  these 
errors  would  likely  be  insignificant.  However,  as  a  further  consider¬ 
ation,  it  was  decided  that  a  validation  procedure  would  be  useful  as 
an  additional  check  on  the  estimation  procedure. 

A  method  for  validation  was  the  primary  motivation  for  the 
development  of  the  exact  procedures  discussed  in  Section  4.1.1.  After 
calculating  exact  levels  of  significance  for  the  2x2  table  for  a 
selected  number  of  sample  sizes,  these  can  be  compared,  in  some  fashion, 
with  the  estimated  levels  from  the  Monte  Carlo  procedure. 

With  the  number  of  tables  generated  for  the  calculation  of  each 

estimate  as  high  as  2000,  the  normal  distribution,  as  noted  in  the 

previous  section,  provides  an  excellent  approximation  to  the  binomial 

random  variable  X  (number  of  rejections  in  K*2000  tables),  where  the 

binomial  parameter  p  is  the  "exact"  level  of  significance.  Then 
1/2 

(X-Kp)/(Kp(l-p))  has  an  approximate  normal  (0,1)  distribution.  Di- 

1/2 

viding  by  K  in  the  numerator  and  denominator,  (X/K-p)/(p(l-p) /K)  has 
an  approximate  normal  (0,1)  distribution.  Forming  a  probability  state¬ 
ment  in  a  manner  similar  to  the  derivation  of  confidence  intervals. 


.■-S.V.V.V.  /./. 
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P(p-zcx/2(p(1-p) /IC) 1/2  <  X/K  <  p  +  Za/2(p(l-p)/K)1/2)  -  1-a.  (4-2) 

These  intervals,  based  on  the  "exact"  levels  of  significance  (p) ,  can 
be  used  to  compare  the  Monte  Carlo  results. 

The  exact  program  for  the  2x2  table  was  run  for  the  15  probabi¬ 
lity  vectors  of  the  complete  design  (Appendix  B)  and  for  sample  sizes 
N  ■  4(4)36  and  40(8)56.  This  data  is  presented  in  Appendix  F.  The 
Monte  Carlo  program  was  run  for  the  complete  probability  design  and 
N  ■  20(4)36  and  40(8)96.  This  provided  an  overlap  area  of  comparable 
data  for  all  15  vectors  and  eight  sample  sizes,  N  *  20(4)36  and  40(8)56 
for  each  of  the  nine  statistic/nominal  level  combinations.  The  over¬ 
lap  Monte  Carlo  data  is  presented  in  Appendix  G.  Intervals  were 
calculated  based  on  (4-2)  using  the  "exact"  levels  of  significance  from 
the  exact  program  for  probabilities,  (1-a),  of  .90  and  .95.  The  Monte 
Carlo  data,  realizations  of  X/K,  were  then  checked  to  see  if  they  fell 
within  these  intervals.  Summary  percentages  of  those  falling  outside 
these  Intervals  are  presented  in  Table  9. 

Before  discussing  the  results  of  the  validation,  it  should  be 
noted  that  each  row  of  Table  9  only  corresponds  to  observations  for  15 
vectors.  Even  though  total  percentages  are  given  for  each  (a,N)  and 
for  each  N,  these  percentages  are  still  based  on  those  same  15  vectors. 
Therefore,  care  should  be  taken  when  drawing  conclusions. 

There  appears  to  be  no  specific  trend  with  respect  to  sample 
size  or  *  eristic.  The  estimates  perform  equally  well  for  each  statis¬ 
tic  as  reflected  in  the  NT0TALS  for  each  (statistic,  a)  combination. 


Table  9.  Monte  Carlo  Validation  (I) 


MONTE  CARLO  VALIDATION  (I) 
PERCENTAGES  OUTSIDE  .95 
PROBABILITY  INTERVALS 


ALPHA  -  .10 


ALPHA  -  .05 


ALPHA  -  .01 


MONTE  CARLO  VALIDATION  (I) 
PERCENTAGES  OUTSIDE  .90 
PROBABILITY  INTERVALS 


NTOTALS  - 


Total  for  each  a 
Grand  Total 


N 

K 

P 

£ 

I 

K 

Z 

£ 

1 

K 

Z 

£ 

1 

NTOTi 

20 

6 

13 

0 

6 

0 

0 

0 

0 

6 

0 

0 

2 

2 

24 

6 

6 

0 

4 

0 

0 

6 

2 

13 

6 

0 

6 

4 

28 

0 

0 

6 

2 

0 

0 

0 

0' 

0 

0 

6 

2 

1 

32 

0 

0 

0 

0 

0 

0 

0 

0 

6 

6 

0 

4 

1 

36 

6 

6 

6 

6 

6 

13 

13 

11 

6 

6 

0 

4 

7 

40 

0 

0 

6 

2 

13 

6 

13 

11 

20 

13 

6 

13 

8 

48 

0 

0 

0 

0 

0 

0 

0 

0 

13 

13 

0 

8 

2 

56 

0 

0 

0 

0 

13 

13 

6 

11 

6 

6 

6 

6 

5 

NTOTALS 

2 

3 

2 

2 

4 

4 

5 

4 

9 

6 

2 

6 

4 

ALPHA 

»  . 

,10 

ALPHA 

-  .05 

ALPHA 

• 

01 

N 

L 

Z 

£ 

I 

K 

P 

G 

T 

K 

P 

G 

T 

NTOTi 

20 

13 

13 

6 

11 

6 

0 

6 

4 

6 

0 

0 

2 

5 

24 

13 

6 

0 

6 

0 

0 

13 

4 

20 

13 

6 

13 

8 

28 

0 

0 

6 

2 

6 

6 

6 

6 

13 

26 

6 

15 

8 

32 

0 

13 

20 

11 

6 

6 

6 

6 

6 

6 

20 

11 

9 

36 

26 

40 

6 

24 

20 

13 

26 

20 

13 

13 

26 

17 

20 

40 

0 

6 

13 

6 

20 

20 

13 

17 

26 

26 

6 

20 

14 

48 

6 

0 

6 

4 

6 

13 

0 

6 

13 

13 

13 

13 

8 

56 

0 

13 

13 

8 

20 

13 

6 

13 

13 

20 

13 

15 

12 

NTOTALS 

7 

11 

9 

9 

10 

9 

10 

10 

14 

15 

11 

13 

11 

K  -  Kullback 

T  - 

Total 

.  for 

each 

(ot,N) 

P  -  Pearson 

NTOTAL  - 

Total  for 

each 

N 

G  -  GSK 

l 

Total  for 

each 

(statistic, a) 

Table  10.  Monte  Carlo  Validation  (II) 


MONTE  CARLO  VALIDATION  (II) 
PERCENTAGES  OUTSIDE  .95 
PROBABILITY  INTERVALS 


ALPHA  »  .10 


4LEBAJ....Q5 


ALP-HA  =  ,Q1 


N 

20 

24 

28 

32 

36 

40 

48 

56 


NTOTALS 


K  P 
6  6 
0  0 
6  0 
0  0 
0  0 
20  6 
6  6 
0  0 

5  2 


G  T 
0  4 
0  0 
6  4 
13  4 
0  0 
6  11 
6  6 
0  0 

4  3 


K  P 
0  0 
20  6 
0  0 
6  13 

6  0 
6  13 

0  0 
6  0 

5  4 


£  T 
0  0 
0  8 
0  .  0 
6  8 
0  2 
6  8 
6  2 
0  2 

2  4 


£  2 
0  0 
13  6 

6  6 
6  0 
6  0 
20  6 
0  0 
6  6 

7  3 


£  2 

13  4 

0  6 
0  4 

13  6 

13  6 

6  11 
0  0 
0  4 


5  5 


NTOTAL 

2 

5 
2 

6 
2 

10 

2 

2 

4 


MONTE  CARLO  VALIDATION  (II) 
PERCENTAGES  OUTSIDE  .90 
PROBABILITY  INTERVALS 


ALPHA  »  .10  ALPHA 


N 

K 

P 

G 

I 

K 

P 

20 

6 

6 

13 

8 

0 

0 

24 

0 

0 

6 

2 

26 

6 

28 

13 

6 

13 

11 

0 

0 

32 

13 

20 

13 

15 

6 

20 

36 

6 

6 

6 

6 

6 

6 

40 

26 

20 

13 

20 

13 

13 

48 

6 

6 

6 

6 

0 

0 

56 

6 

6 

0 

4 

6 

6 

NTOTALS 

10 

9 

9 

9 

7 

6 

■  . 

£5 

ALPHA 

=  . 

01 

£ 

1 

K 

P 

G 

T 

NTOTAL 

0 

0 

6 

13 

13 

11 

6 

0 

11 

13 

13 

6 

11 

8 

6 

2 

6 

6 

6 

6 

6 

26 

17 

13 

6 

13 

11 

14 

6 

6 

6 

6 

20 

11 

8 

6 

11 

20 

20 

13 

17 

16 

6 

2 

6 

0 

0 

2 

3 

0 

4 

6 

6 

20 

11 

6 

6 

6 

10 

9 

11 

10 

8 

K  -  Kullback 
P  -  Pearson 
G  -  GSK 


T  -  Total  for  each  (ot,N) 
NTOTAL  -  Total  for  each  N 


NTOTALS  - 


Total  for  each  (statistic,!*) 
Total  for  each  a 
Grand  Total 
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These  NTOTALS  are  based  on  the  15  vectors  and  eight  sample  sizes,  or 
120  observations.  The  NTOTAL  for  each  sample  size  indicates  a  highly 
irregular  pattern,  which  should  be  expected  with  only  15  observations 
if  the  procedure  is  performing  properly.  There  does  appear  to  be  a 
trend  with  respect  to  the  nominal  alpha  levels.  As  the  nominal  alpha 
decreases  the  percentages  increase.  This  might  be  expected  since  these 
intervals  were  formed  based  on  the  normal  approximation  to  the  binomial. 
This  approximation  is  known  to  be  less  acurate  for  smaller  values  of  p. 
The  grand  total  percentages  of  four  and  11  for  the  respective , .95  and 
.90  probability  intervals  correspond  very  well  with  what  is  expected. 

There  was  concern  for  some  of  the  relatively  high  percentage?, 
particularly  for  the  sample  size  N-36  where  the  NTOTAL  percentage  was 
20  for  the  .90  probability  interval.  It  was  decided  to  replicate  the 
Monte  Carlo  simulation  for  these  same  overlapping  sample  sizes.  Table 
10  provides  the  summary  results.  The  NTOTAL  percentage  for  N*36  is 
eight  for  the  .90  probability.  It  is  obvious  that  for  the  first  valida¬ 
tion  run,  significantly  more  outliers  affected  the  N*36  calculations. 
Considering  the  data  in  Table  10,  as  well  as  the  data  in  Table  9,  there 
are  still  no  apparent  trends  with  respect  to  sample  size  or  statistic. 
The  nominal  alpha  trend  noted  in  Table  9  is  not  as  clear  in  Table  10. 
There  is  an  actual  decrease  in  percentages  from  the  nominal  alpha  .10 
to  the  nominal  alpha  .05  for  the  .90  probability  intervals.  The  grand 
total  percentages  of  four  and  eight  for  the  respective  .95  and  .90 
probability  intervals  again  are  very  good. 

In  summary,  these  validation  results  indicate  that  no  trends 
exist  with  respect  to  the  parameters  of  the  simulation,  and  that  the 
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simulation  provides  reasonable  results.  However,  it  is  apparent  that 
care  should  be  taken  when  drawing  conclusions  from  the  data  since 
severe  irregularities  will  occur  due  to  outliers. 
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CHAPTER  V 

DATA  ANALYSIS 

This  chapter  will  present  a  detailed  analysis  of  the  data  from 
the  exact  program  and  the  Monte  Carlo  simulation.  The  data  directly 
from  these  programs  will  be  called  the  "basic"  data.  The  exact  2x2 
basic  data  are  presented  in  Appendix  F.  The  complete  data  for  the  2x3 
table  provide  examples  of  the  Monte  Carlo  basic  data  and  are  presented 
in  Appendix  H.  The  presentation  of  this  data  will  be  discussed,  followed 
by  a  comparison  with  other  studies  and  a  brief  analysis  with  respect  to 
general  trends.  From  this  basic  data,  minimum  and  maximum  significance 
levels  for  minimum  cell  expectation  intervals  (MCEI)  will  be  reported 
for  each  of  the  table,  hypothesis,  statistic,  and  nominal  level  combi¬ 
nations.  These  levels,  presented  in  Appendix  I,  will  be  analyzed  as  to 
general  trends  with  particular  concern  for  possible  results  across 
tables.  Returning  to  the  basic  data,  graphical  plots  of  this  data 

will  be  used  to  determine  certain  "minimum  N"  (N  )  based  on  a  confidence 

m 

interval  criterion.  These  N^,  presented  in  Appendix  J,  will  provide  the 
parameters  for  calculating  Critical  Expected  Value  (CEV)  Distributions. 
These  CEV  distributions,  presented  in  Appendix  K,  will  be  extensively 
analyzed.  Comparisons  will  be  made  of  the  performance  of  the  statistics, 
and  trends  indicated  across  tables.  A  useful  parameter  will  be  dis¬ 
cussed  along  with  other  attempts  of  parametric  analysis. 


5.1  Basic  Data 


The  presentation  of  the  basic  data  (Appendices  F  and  H)  includes 
the  underlying  probability  vector  (P  VECTOR) ,  the  sample  size  (N) ,  and 
the  exact  or  estimated  exact  levels  of  significance  for  the  Kullback, 
Pearson,  and  GSK  statistics  at  the  nominal  alpha  levels  of  significance 
(.10,  .05,  .01).  Each  probability  vector  was  used  with  all  sample 
sizes  of  the  design  in  Appendix  C, and  the.  vectors  are  arranged  in  the 
order  of  the  scheme  given  in  Appendix  5.  This  section  will  first  dis¬ 
cuss  some  comparisons  of  the  basic  data  with  previous  studies  and  then 
present  a  brief  of  analysis  of  some  apparent  trends. 

5.1.1  Comparisons  with  Previous  Studies 

As  discussed  in  Chapter  II,  previous  studies  were  somewhat  limited 
in  scope,  particularly  with  respect  to  the  underlying  probability 
structure.  However,  comparisons  can  be  made  with  the  study  of  this 
thesis  for  the  basic  data  at  certain  points  of  the  design.  These  com¬ 
parisons  are  useful  in  further  validating  the  procedure  of  this  study 
and  providing  information  for  comparing  conclusions. 

The  first  study  that  provides  some  points  for  comparison  is 

Haynam  and  Leone  (1965).  They  presented  graphs  for  the  exact  distribu- 

2 

tlon  of  the  Pearson  X  statistic  when  used  in  testing  independence  for 
2x3  and  3  *  3  tables  for  the  multinomial  sampling  model,  equiprobable 
vector,  and  sample  sizes  of  10  and  15.  Using  the  Haynam  and  Leone 
graphs,  Table  11  shows  comparison  points  with  the  basic  data  from  the 
Monte  Carlo  programs  of  this  study.  It  should  be  noted  that  Haynam 
and  Leone  did  not  specify  their  procedures  for  handling  zero  marginals; 
although,  for  the  equiprobable  table  the  probability  of  zero  marginals 
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Table  11.  Comparisons  with  Haynam  and  Leone  Data 


2x3  Table 

Nominal  a 

Haynam  and 

Leone 

Kolb 

.01 

N-10 

.004 

N-12 

.0025 

N-15 

.006 

N-18 

.0085 

.05 

N-10 

.048  . 

N-12 

.0490 

N-15 

.050 

N-18 

.0465 

.10 

N-10 

.100 

N-12 

.0975 

N-15 

.100 

N-18 

.0970 

3x3  Table 

Nominal  a 

Haynam  and 

Leone 

Kolb 

.01 

N-10 

.004 

N-15 

.006 

N-18 

.0060 

.05 

N-10 

.035 

N-15 

.040 

N-18 

.0390 

.10 

N-10 

.075 

N-15 

.095 

N-18 

.0950 

for  the  2x3  and  3x3  tables  would  be  relatively  small.  Table  11 
demonstrates  the  close  agreement  between  the  exact  levels  of  signifi¬ 
cance  and  the  estimated  levels  from  the  Monte  Carlo  procedure  of  the 
present  study. 

Craddock  (1966)  provided  estimated  percentiles  for  the  Pearson 
statistic  under  the  independence  hypothesis  in  3  x  3  equiprobable  tables, 
based  on  a  Monte  Carlo  of  10,000  values  with  sample  sizes  ranging  up 
to  100.  Although  exact  comparisons  are  not  possible  due  to  the  form  of 
presentation,  comparisons  can  be  made  as  to  whether  the  statistic  is 


conservative  or  liberal  at  the  particular  sample  points.  Table  12 
provides  these  comparisons.  Only  at  the  .05  level  are  discrepancies 


Table  12. 

Comparisons  with 

Craddock 

Data 

3x3  Table 

Craddock 

Kolb 

Nominal 

a 

Nominal 

a 

N 

.10 

.05 

.01 

N 

.10 

.05 

.01 

15 

C 

C 

C 

18 

C 

C 

C 

20 

- 

c 

c 

27 

L 

L 

c 

25 

L 

c 

c 

36 

L 

L 

c 

30 

L 

c 

c 

108 

L 

C 

L 

100 

L 

L 

L 

L  - 

Liberal 

C  - 

Conservative 

mm  mm 

Equal 

noted.  Upon  examination  of  the  specific  data  at  these  points,  the  es ti' 

mates  are  so  close  to  the  nominal  levels  that  discrepancies  can  be 

interpreted  to  be  the  result  of  sampling  error. 

Craddock  and  Flood  (1970)  extended  Craddock's  study,  including 

tables  from  2x3  to  5x5.  They  provided  similar  estimated  percentiles 
2 

for  the  Pearson  X  .  Table  13  reflects  the  comparisons  with  the  present 
study.  The  percentiles  given  by  Craddock  and  Flood  were  only  estimated 
to  one  decimal  place,  so  comparisons  with  the  present  study  are  diffi¬ 
cult  when  exact  levels  are  relatively  close  to  nominal  levels.  Upon 
examination  of  those  points  considered  exact  (-)  by  Craddock  and  Flood, 
the  largest  error  for  the  present  study  is  seven  percent  of  the  nominal 


level,  and  all  estimates  are  well  within  a  95  percent  confidence  inter¬ 
val  about  the  nominal  level.  There  are  then  only  13  out  of  57  points 
with  apparent  discrepancies,  but  upon  closely  examining  these  points 
the  discrepancies  are  again  apparently  due  to  sampling  error. 

Miller  (1979)  estimated  exact  significance  levels  for  both  the 
Pearson  and  Kullback  statistics  for  selected  tables,  probability  vectors, 
and  sample  sizes  based  on  a  Monte  Carlo  study  with  2000  tables.  Those 
values  comparable  with  the  present  study  are  given  in  Table  14.  Com¬ 
parisons  for  the  2  *  2  table  for  small  N  were  excluded  since  exact  values 


sidered  carefully.  Even  with  only  these  few  comparisons  it  should  be 
evident  that  there  is  a  consistency  with  respect  to  the  conservative 
or  liberal  nature  of  these  estimates  when  the  exact  values  are  clearly 
on  one  side  of  the  nominal  levels.  It  should  also  be  evident  that 
when  the  estimate  is  near  the  nominal  level,  comparisons  cannot  be 
reasonably  made. 

5.1,2  Analysis 

The  basic  data  in  Appendix  II  are  representative  of  the  data 
from  the  Monte  Carlo  simulation.  Directly  from  the  basic  data,  several 
interesting  observations  can  be  made.  First,  as  should  be  expected,  due 
to  the  asymptotic  nature  of  these  statistics,  the  general  trend  as  the 
sample  size  N  increases  is  for  the  estimated  exact  levels  of  significance 
to  approach  the  nominal  levels.  However,  the  statistics  each  behave 
differently  with  respect  to  their  asymptotic  nature. 

The  Kullback  statistic,  at  selected  probability  vectors,  actually 
appears  to  depart  from  the  general  asymptotic  trend.  In  fact,  for  some 
of  the  more  extreme  vectors  (more  skewed  away  from  equiprobable)  the 
estimated  exact  levels  actually  diverge  from  the  nominal  levels.  Con¬ 
sidering  the  2x3  data  in  Appendix  H,  for  the  fourth  probability  vector, 
(.02,  .04,  .04,  .18,  .36,  .36),  at  N-12  the  estimated  exact  levels  are 
.0915  and  .0410  for  the  .10  and  .05  nominal  levels,  respectively.  These 
values  are  within  95  percent  confidence  intervals  about  the  nominal 
levels.  At  N-60  the  estimates  are  .1465  and  .0730,  well  outside  the 
confidence  intervals.  At  N*96  the  estimates  are  .1245  and  .0665,  still 
outside  the  confidence  intervals.  Figure  13  provides  a  graph  of  the 
estimates  for  the  Kullback  statistic  under  the  probability  vector.  The 
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graph  demonstrates  a  quadratic  effect  for  these  estimates.  For  small 
N  the  estimates  begin  relatively  close  to  the  nominal  levels,  and 
then  move  farther  away  as  N  increases.  At  some  N  the  estimates  begin 
to  move  back  toward  the  nominal  levels.  For  the  more  extreme  vectors 
this  quadratic  effect  seems  fairly  consistent  for  all  the  tables  and 
hypotheses  considered.  An  additional  general  observation  for  the 
Kullback  statistic  is  that  it  tends  to  he  liberal.  This  observation 
agrees  with  the  results  from  previous  studies  for  the  log-likelihood 
ratio  statistic. 

The  GSK  statistic  is  consistently  conservative.  In  fact,  for 
the  more  extreme  vectors  and  larger  tables  the  statistic  is  so  conserva¬ 
tive  that  its  value  as  an  independence  hypothesis  test  statistic  under 
the  log-linear  model  is  highly  questionable.  For  these  cases  the  power 
of  the  test  would  be  so  low  that  the  null  independence  hypothesis  would 
not  be  rejected  even  for  extreme  departures  from  independence.  The 
GSK  statistic  does  not  begin  to  give  reasonable  results  until  near  the 
equiprobable  vector,  and  then  only  for  large  values  of  N. 

One  of  the  problems  associated  with  the  GSK  statistic  is  the 
effect  of  zero  observed  cells  as  discussed  in  Section  4.1.2.  The  method 
for  handling  these  zero  cells  was  to  add  1/r  (r  equal  to  the  number  of 
cells)  to  each  zero  cell.  This  method  was  chosen  because  it  is  the  most 
popular.  Little  is  known  about  the  specific  bias  effect  of  this  method 
or  any  of  the  other  suggested  methods.  It  is  known  that  the  bias  tends 
to  make  the  statistic  more  conservative.  A  case  could  be  made  that 
this  is  the  reason  for  the  ultra-conservative  nature  of  the  GSK  statis¬ 
tic,  demonstrated  by  the  results  of  this  study.  This  is  probably  valid 
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for  the  extreme  vectors,  where  the  probability  of  having  sampling 
zeroes  is  relatively  high,  but  this  should  not  significantly  affect 
the  more  equiprobable  vectors.  Table  15  provides  some  conditional 
probabilities  for  obtaining  a  zero  observed  cell,  given  the  no  zero 
marginal  sampling  procedure  of  this  study.  From  this  table  it  is 
obvious  that  a  zero  cell  bias  could  significantly  affect  the  estimates 
at  the  more  extreme  vectors,  but  should'  not  significantly  affect  the 
estimates  at  the  more  equiprobable  vectors. 


Table  15.  Conditional  Zero  Cell  Proabilities 


Table 

P  VECTOR 

N 

Probability 

2x2 

(.01, .09, .09, .81) 

20 

.8366 

56 

.5645 

96 

.3810 

(1/4, 1/4, 1/A, 1/4) 

20 

.0127 

56 

4.03  x 10-7 

96 

4.05  x 10-12 

2x3 

(.01, .01, .08, .09, .09, .72) 

24 

.9608 

54 

.8098 

96 

.6185 

(1/6, 1/6, 1/6, 1/6, 1/6, 1/6) 

24 

.0746 

54 

.0003 

96 

1.5  x  10~7 

The  basic  data  for  the  Pearson  statistic  indicate  that  the 
statistic  performs  considerably  different  than  either  the  Kullback  or 
GSK  statistic.  With  respect  to  the  closeness  of  exact  and  nominal 
levels,  the  Pearson  statistic  is  clearly  superior  to  either  of  the 
other  statistics,  but  it  is  also  more  erratic  as  to  a  conservative  or 
liberal  nature.  For  vectors  close  to  the  equiprobable  the  general 
trend  of  the  statistic  is  to  begin  conservatively  and  move  toward  the 
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nominal  level  as  N  increases.  For  the  more  extreme  vectors  the  general 
trend  is  to  begin  liberally  and  move  toward  the  nominal  levels.  Excep¬ 
tions  to  these  trends  do  exist.  In  particular,  for  the  most  extreme 
vector,  (.01,  .01,  .08,  .09,  .09,  .72),  of  the  2x3  table  at  the  nominal 
.05  level,  the  statistic  shows  a  quadratic  effect  similar  to  the  Kullback 
The  statistic  begins  conservatively  and  moves  away  from  the  nominal  level 
then  turns  abruptly  (at  N*36)  back  towatd  the  nominal  level.  Also,  at 
small  values  of  N  the  statistic  tends  to  be  more  erratic  than  the  other 
statistics,  but  relatively  quickly  converges  toward  the  nominal  levels. 
The  general  superior  performance  of  the  statistic,  as  reflected  in  the 
basic  data,  corresponds  to  conclusions  of  other  studies  when  comparing 
the  Pearson  and  log- likelihood  ratio  statistics. 

5.2  Minimum  Cell  Expectation  (MCE) 

One  of  the  controversies  surrounding  the  use  of  these  asymptotic 
statistics  has  been  how  important  minimum  cell  expectation  (MCE)  is  with 
respect  to  the  closeness  of  the  chi-squared  distribution  to  the  exact 
distributions  of  these  statistics.  The  regression  example  in  Chapter  I 
demonstrates  the  problems  associated  with  small  expected  values,  but 
in  that  example  all  the  expected  values  become  small. 

5.2.1  Previous  Results 

Several  authors  have  indicated  that  the  MCE  alone  is  significant; 
in  particular,  both  Cramer  (1946)  and  Cochran  (1952)  emphasized  the 
importance  of  the  MCE.  Almost  all  the  "rules  of  thumb"  given  for  the 
use  of  the  Pearson  statistic  are  given  in  terms  of  MCE.  However,  very 
little  empirical  evidence  is  available  that  directly  relates  MCE  and 


the  performance  of  any  of  these  statistics  for  contingency  table  anal¬ 
ysis. 

Yamold  (1970)  did  present  some  empirical  evidence  for  the 
Pearson  statistic  when  used  as  a  multinomial  goodness-of-fit  test. 

Here  he  provided  his  5r/s  rule.  This  study  was  somewhat  contradicted 
by  a  study  by  Tate  and  Hyer  (1973),  where  they  concluded  that,  "general 
statements  about  minimum  expectation  are  of  limited  usefulness".  Only 
Odoroff's  study  provided  some  indication  of  the  role  of  MCE  with  respect 
to  contingency  table  tests.  However,  his  MCE  was  not  the  usual  contin¬ 
gency  table  expectation  computed  from  the  observed  table.  His  MCE  was 
based  on  choosing  the  parameters  of  his  model  so  that  the  smallest  cell 
expectation  was  constant.  His  results  did  indicate  general  improve¬ 
ment  as  MCE  increased  for  most  of  the  statistics  he  evaluated,  but  his 
results  were  restricted  to  the  specialized  framework  of  his  model  and 
hypothesis  in  the  2><2x2  and  3x2x2  tables  with  fixed  and  equal  one¬ 
way  marginals.  None  of  the  previous  studies  provided  information  regard¬ 
ing  MCE  and  general  independence  test  statistics. 

As  a  means  of  comparing  three  competing  multinomial  goodness-of- 
fit  statistics,  Larntz  (1978)  used  a  very  informative  summary  table. 

This  table  showed  the  minimum  and  maximum  significance  levels  for  nomi¬ 
nal  .05  tests  using  data  from  Monte  Carlo  sampling.  These  levels  were 
reported  based  on  the  number  of  cells  of  the  multinomial  (2,  3,  5,  10) 
and  intervals  of  minimum  cell  expectations.  His  results  indicated 
general  improvement  of  the  statistics  as  MCE  increased  and  provided  a 
means  for  comparing  the  statistics. 
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5.2.2  Tabulation  of  Minimum  and  Maximum  Significance  Levels  for 

Minimum  Cell  Expectation  Intervals  (MCEI) 

Using  the  same  procedure  as  Larntz,  the  basic  data  from  the  study 
of  this  thesis  were  used  to  tabulate  minimum  and  maximum  significance 
levels  for  selected  minimum  cell  expectation  intervals  (MCEI) .  These 
MCEI  were  half-open/half-closed  intervals,  from  [0.5,  1.0)  to  [4.5, 

5.0),  each  of  width  0.5.  The  levels  were  determined  by  considering 
each  probability  vector  and  sample  size  design  point,  then  categorizing 
it  into  one  of  the  MCEI  by  using  the  smallest  cell  probability  and  mul¬ 
tiplying  it  by  the  sample  size.  The  levels  for  each  of  the  statistic 
and  nominal  level  combinations  were  then  compared  against  previous 
minimum  and  maximum  levels  for  the  corresponding  MCEI.  For  each  design 
point, minimum  and  maximum  significance  levels  were  updated  as  necessary 
until  all  design  points  were  considered.  These  levels  are  presented 
in  Appendix  I  for  each  of  the  table  and  hypothesis  combinations  and 
each  of  the  statistic  and  nominal  level  combinations. 

5.2.3  Analysis 

With  respect  to  these  levels,  performance  of  the  statistics  is 
measured  by  how  close  the  minimum  and  maximum  levels  are  to  the  nominal 
levels.  Even  a  cursory  look  at  these  levels  shows  immediately  that, 
in  general,  the  statistics  perform  better  as  the  MCE  increases.  Gen¬ 
eralizations  for  the  Individual  statistics  follow  very  closely  those  of 
the  previous  section  with  respect  to  sample  sizes. 

The  interval  formed  by  the  minimum  and  maximum  significance 


levels  (MMSLI)  for  the  Kullback  statistic  generally  includes  the  nomi¬ 
nal  level  at  the  lowest  MCEI,  [0.5,  1.0).  As  the  MCEI  increase,  the 
MMSLI  generally  become  smaller,  but  exclude  the  nominal  level  since  the 


minimum  levels  become  larger  Chan  Che  nominal  levels.  This  indicaCes 
Che  very  liberal  naCure  of  Che  Kullback  statistic. 

The  MMSLI  for  the  GSK  statistic  are  all  relatively  small,  but 
the  maximum  level  is  always  smaller  than  the  nominal  level.  For  lower 
MCEI  the  maximum  levels  are  considerably  smaller  than  the  nominal 
levels.  This  reflects  the  ultra-conservative  nature  of  the  GSK  static- 
tic.  There  is  an  obvious  improvement  in  the  statistic  as  the  MCE 
increases. 

The  MMSLI  for  the  Pearson  statistic  are  relatively  large  for 
the  lower  MCEI,  but  do  include  the  nominal  level.  This  reflects  the 
erratic  nature  of  the  statistic  for  very  small  MCE.  The  MMSLI  narrow 
as  the  MCEI  increase  and,  in  general,  they  continue  to  include  the 
nominal  level.  For  the  higher  MCEI  both  the  minimum  and  maximum  levels 
are  generally  both  within  a  95  percent  confidence  interval  of  the  nomi¬ 
nal  levels.  This  is  not  true  for  either  the  Kullback  or  GSK  statistics. 
Previous  studies  have  indicated  that  the  Pearson  statistic  could  be 
used  with  expected  values  smaller  than  five.  The  results  here  further 
support  this  claim. 

The  more  important  aspect  of  this  study  is  to  investigate  the 
nature  of  these  statistics  across  tables.  These  minimum  and  maximum 
significance  levels  do  provide  valuable  comparison  information  for  the 
statistics,  but  trends  across  tables  seem  difficult  to  interpret. 

Only  the  levels  for  the  GSK  statistic  indicate  a  trend.  For  a  given 
MCEI,  in  general,  these  levels  move  farther  away  from  the  nominal  levels 
(at  .10  and  .05)  as  the  size  of  the  table  increases.  This  indicates 
a  general  degradation  in  the  performance  of  the  GSK  statistic  with 
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respect  to  MCE  as  the  number  of  cells  increases* 

From  the  results  here,  MCE  is  obviously  an  important  factor  in 
the  performance  of  these  statistics.  However,  it  does  not  appear  to 
provide  significant  information  to  draw  conclusions  across  tables, 
particularly  for  the  Pearson  and  Kullback  statistics.  Also,  with 
larger  contingency  tables,  it  would  seem  unlikely  that  just  a  con¬ 
sideration  of  one  cell  would  be  sufficient  to  measure  the  performance 
of  a  statistic.  The  next  section  will  provide  a  means  for  finding 
trends  across  tables. 


5.3  Critical  Expected  Value  (CEV)  Distributions 


A  significant  aspect  of  any  empirical  study  is  a  determination 
of  appropriate  parameters  and  "criteria"  for  measuring  effectiveness. 
From  the  previous  section  it  was  seen  that  MCE  alone  did  not  provide 
sufficient  information  for  measuring  effectiveness  in  relation  to  the 
primary  purpose  of  this  study.  This  section  will  provide  a  criterion 
for  measuring  this  effectiveness  in  the  form  of  critical  expected  value 
(CEV)  distributions. 


5.3.1  Minimum  Sample  Size  (N  ) 

. . .  Q  — 

The  primary  concern  regarding  thes> 


'sties  is  their  asymp¬ 


totic  nature;  that  is,  given  a  specified  unu»-.  -  ag  probability 
structure  based  on  a  null  hypothesis  of  independence,  when  does  the 
statistic  begin  to  perform  well.  Here  the  answer  is  very  subjective, 
and  probably  no  single  criterion  would  satisfy  all  situations.  While 
this  study  is  primarily  concerned  with  discovering  trends  empirically 
under  specified  conditions,  care  needs  to  be  taken  to  establish  para¬ 
meters  and  criteria  that  seem  reasonable  and  that  will  provide  trend 


information 


Since  these  statistics  are  asymptotic,  in  that  their  exact 
distributions  approach  the  chi-squared  distribution  as  the  sample 
size  (N)  Increases,  N  would  be  a  reasonable  parameter.  As  far  as  a 
criterion  for  performance,  some  measure  of  closeness  to  the  nominal 
level  of  significance  would  seem  appropriate.  The  95  percent  confi¬ 
dence  intervals;  based  on  the  Monte  Carlo,  sampling  of  2000  tables,  the 
normal  approximation  to  the  binomial,  and  the  nominal  (.10,  .05,  .01) 
levels  of  significance;  provide  convenient  and  reasonable  measures  of 
"closeness"  to  these  nominal  values.  These  intervals,  calculated 
exactly  as  the  accuracy  levels  in  Section  4.3.2,  are  (.0869,  .1131), 
(.0404,  ,0596),  and  (.0056,  .0144)  for  the  nominal  .10,  .05,  and  .01 
levels,  respectively.  The  performance  of  a  statistic  will  be  consider¬ 
ed  good  when  the  estimated  exact  levels  are  within  these  intervals, 
which  are  centered  on  the  nominal  levels.  This  criterion  is  intuitively 
appealing  since  a  hypothesis  test  on  the  nominal  level  would  not  be  re¬ 
jected  if  the  estimated  exact  level  was  within  this  interval. 

The  above  criterion  assumes  that  both  overestlmatlon  and  under¬ 
estimation  of  the  nominal  levels  are  of  equal  importance.  From  the 
discussion  in  Section  4.1.1  this  would  seem  reasonable  since  it  pro¬ 
vides  equal  consideration  of  both  a  conservative  test  and  a  more 
powerful  test.  Using  intervals  of  this  nature  is  not  without  prece¬ 
dence.  Cochran  (1952)  suggested  that  a  "disturbance"  is  unimportant 
if  it  lies  within  the  intervals  (.04,  .06)  and  (.007,  .015)  for  the 
nominal  .05  and  .01  levels,  respectively.  Roscoe  and  Byars  (1971)  used 
Cochran's  intervals.  Odoroff  (1970)  and  Wang  (1979)  also  used  (.04, 


.06)  for  the  nominal  .05  level.  Wang  chose  an  "acceptable"  interval 
for  the  nominal  .01  level  of  (.005,  .015).  Larntz  (1978)  used  the 
Interval  (.03,  .07)  for  the  nominal  .05  level.  Which  specific  inter¬ 
vals  are  used  is  rather  immaterial  as  long  as  they  provide  a  means  for 
comparing  the  performance  of  these  statistics.  It  is  unlikely  that 
choosing  any  of  the  above  intervals  would  lead  to  significantly  different 
conclusions. 

Once  the  intervals  are  selected,  the  goal  is  to  determine  for  each 
statistic,  table,  hypothesis,  nominal  level,  and  probability  vector  com¬ 
bination  the  minimum  sample  size  (N^)  where  the  exact  significance 
levels  are  within  the  95  percent  confidence  intervals  and  stay  within 
these  Intervals  as  the  sample  size  increases. 

To  assist  in  the  determination  of  these  N  ,  graphs  of  the  basic 

m 

data  were  used.  Figures  14,  15,  and  16  are  the  graphs  for  the  Kullback, 
Pearson,  and  GSK  statistics,  respectively,  for  the  2><3  table  and 
fifteenth  (P15)  probability  vector  of  the  design  in  Appendix  B.  Each 
graph  plots  the  estimated  exact  significance  levels  for  each  value  of  N. 
Each  figure  contains  three  graphs  representing  the  three  nominal  levels 
using  the  following  symbols: 

*  -  .10  nominal  level 
x  -  .05  nominal  level 
+  -  .01  nominal  level 


Straight  lines  a ?e  used  to  connect  the  points  but  have  no  special  sig¬ 
nificance.  They  are  used  only  as  a  visual  guide  to  help  identify  out- 
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Figure  14.  Kullback,  2*3,  Probability  Vector  Ho.  15 
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Pearson,  2x3,  Probability  Vector 


liers  and  determine  the  N  . 
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The  procedure  used  to  determine  these  Nq  was  consistent  and 

allowed  for  resampling  of  suspected  outliers,  particularly  when  these 

suspected  outliers  would  cause  choices  of  N  which  seemed  drastically 

n 

out  of  pattern  with  those  around  them.  It  should  be  noted  that  resam¬ 
pling  did  not  always  substantiate  these  points  as  outliers.  In  those 
cases  the  points  were  considered  valid,'  and  the  resulting  reported. 
Fortunately,  few  of  these  situations  occurred.  The  basic  procedure  Is 
outlined  below: 

1.  Fit  the  95  percent  confidence  interval  about  the  nominal 
level  (Figure  17  -  95  percent  confidence  Interval  template). 

2.  Observe  the  pattern  of  data  and  note  if  there  are  any  sig¬ 
nificant  perturbations. 

3.  If  the  data  points  are  all  outside  the  confidence  Interval, 
report  "greater  than  the  maximum  N”  (e.g.,  >96  for  the  2*2  table). 

4.  If  the  data  points  are  all  Inside  the  confidence  interval, 
report  "less  than  the  minimum  N"  (e.g.,  <12  for  the  2x3  table). 

5.  If  the  data  Is  relatively  smooth,  crossing  into  the  confi¬ 
dence  interval  at  fairly  definitive  points  and  staying  within  this 
interval,  report  this  crossing  sample  size  (approximately  the  linear 
interpolation  between  the  last  point  outside  the  interval  and  the  first 
point  inside  the  interval). 

6.  If  the  data  is  not  smooth  and  crossover  points  are  not  clear 
indicate  suspected  outliers.  Compare  the  data  patterns  with  the  pat¬ 
terns  of  other  statistics  under  the  same  probability  vector  and,  con¬ 
sequently,  based  on  the  same  Monte  Carlo  sample.  Designate  suspected 


outliers  for  resampling. 

7.  After  steps  one  through  six  have  been  performed  for  each 
vector,  statistic,  and  nominal  level  combination  of  a  table,  review  to 
see  if  any  appear  significantly  out  of  pattern.  If  so,  see  if  re¬ 
sampled  points  could  clarify  these  determinations.  Designate  further 
points  for  resampling. 

8.  Resample  all  points  designated  in  steps  six  and  seven. 

9.  Average  the  resampled  data  with  the  basic  data  and  return  to 
step  one  until  all  N  determinations  have  been  satisfactorily  made. 

The  outliers  in  step  six  were  usually  easy  to  determine.  Figures 
14,  15,  and  16  provide  a  typical  example.  The  point  N-60  appears  to 
be  an  outlier  for  all  three  statistics.  Upon  resampling,  the  estimated 
significance  levels  were  considerably  higher.  Only  one  resampling  run 
was  needed  for  each  table.  For  almost  all  cases  suspected  outliers  were 
confirmed,  and  resampling  smoothed  the  data  curves,  allowing  for  reason¬ 
able  N  determinations, 
m 

A  problem  did  occur  with  the  extreme  vectors  for  the  Kullback 
statistic.  It  appeared  that  for  some  extreme  vectors  the  exact  levels 
of  significance  were  within  the  confidence  intervals  for  relatively 
small  N  and  would  stay  within  these  intervals.  The  actual  situation  was 
that  these  exact  levels  were  drifting  out  of  the  confidence  intervals, 
and  the  sample  sizes  were  not  large  enough  to  detect  this  slow  drift. 
This  was  verified  for  several  of  the  extreme  vectors  by  running  the 
Monte  Carlo  programs  for  these  vectors  with  larger  sample  sizes.  Table 
16  provides  an  example  of  this  problem  for  the  2><A  table  and  an 
extreme  vector  at  the  .05  nominal  level.  From  the  original  data  sampled 


Table  16.  Kullback  Extreme  Vector  Example 


2x4  Table 


Vector  »  (.01,  .01,  .03,  . 

05,  .09,  .09, 

.27, 

Nominal 

a  *  .05 

a  «*  estimated  exact  alpha 

X 

A 

“e 

N 

A 

ae 

16 

.0340 

80 

.0510 

24 

.0255 

'  96 

.0585 

32 

.0365 

112 

.0615 

40 

.0370 

144 

.0575 

48 

.0470 

176 

.0640 

64 

.0490 

208 

.0720 

up  to  N»96,  it  would  be  easy  to  conclude  that  N  occurred  near  N=43. 

in 

However,  with  the  extra  data  points  it  is  clear  that  N  is  greater 

m 

than  96  and  even  greater  than  204.  Other  extreme  vectors  followed  a 

similar  pattern,  and  designating  N  >  96  for  these  extreme  vectors 

m 

corresponded  to  the  general  trend  of  larger  for  more  extreme  vectors. 

Appendix  J  provides  tables  of  these  N  for  each  of  the  contin- 

m 

gency  tables  and  hypothesis  tests  considered.  These  N  are  listed  in 

m 

the  probability  vector  order  designated  in  Appendix  B.  For  these  N 

m 

determinations  the  superiority  of  the  Pearson  statistic  is  obvious, 

and  the  Kullback  statistic  is  clearly  better  than  the  GSK.  However, 

the  primary  purpose  of  these  N  is  not  for  comparison  of  the  statistics 

m 

but  to  provide  parameters  for  the  calculation  of  the  critical  expected 
value  (CEV)  distributions. 

5.3.2  Calculation  of  CEV  Distributions 


Minimum  N  determinations  alone  are  not  useful  to  establish 


patterns  across  tables.  A  measure  is  needed  to  determine  how  well  these 


statistics  perform  with  respect  to  small  expected  values,  that  can  be 
compared  across  tables.  Using  '.he  minimum  N  in  Appendix  J,  a  series 
of  critical  expected  value  (CEV)  distributions  were  calculated  and 
are  presented  in  Appendix  K.  To  calculate  these  CEV  distributions,  the 
for  a  probability  vector  was  multiplied  by  each  of  the  cell  proba¬ 
bilities.  The  corresponding  expected  values  were  then  accumulated 
based  on  the  number  of  expected  values  less  than  the  Integer  values 
one  through  10. 

Each  table  of  Appendix  K  presents  the  cumulative  distribution  of 
expected  values  at  the  minimum  N  for  a  given  statistic,  table,  hypo¬ 
thesis,  and  nominal  significance  level  combination  using  all  the  proba¬ 
bility  vectors  of  the  design.  The  distributions  in  the  upper  part  of 
each  table  are  the  "most  extreme"  expected  value  distributions  for  the 
probability  vectors  where  the  exact  significance  levels  (as  estimated 
by  the  Monte  Carlo  procedure)  of  the  statistics  are  within  the  95  per¬ 
cent  confidence  interval  of  the  nominal  level.  "Most  extreme"  implies 
that  the  exact  significance  levels  of  the  statistics  for  all  less 
extreme  distributions  for  these  probability  vectors  are  also  within 
the  95  percent  confidence  interval.  The  lower  part  of  each  table 
corresponds  to  minimum  N  values  that  were  determined  to  be  greater  than 
the  maximum  N  used  in  the  simulation.  These  distributions  represent 
expected  value  distributions  where  exact  significance  levels  of  the 
statistic  were  not  within  the  95  percent  confidence  intervals. 

5.3.3  Analysis 


As  will  be  discussed,  summary  parameters  appear  useless  with 
respect  to  comparisons  across  tables.  The  CEV  distributions  do  provide 


a  convenient  method  for  comparing  the  performance  of  these  statistics 
with  respect  to  small  expected  values  across  tables.  Before  dis¬ 
cussing  these  comparisons,  it  should  be  re-emphasized  that  these  dis¬ 
tributions  were  derived  using  the  95  percent  confidence  interval 
criterion.  The  upper  part  of  the  tables  present  the  most  extreme 
expected  value  distributions  that  meet  this  criterion  where  all  other 
less  extreme  distributions  also  meet  the  criterion.  These  distribu¬ 
tions  then  reflect  the  "best"  expected  value  structures  (smallest 
expected  values  and  largest  number  of  small  expected  values)  where  the 
statistic  satisfies  the  given  criterion.  The  lower  part  of  the  tables 
specifies  expected  value  structures  where  the  statistic  does  not  satis¬ 
fy  the  criterion.  An  advantage  that  these  tables  have  over  the  MCE 
criteria  is  that  they  reflect  not  only  the  smallest  expected  value  but 
also  the  entire  expected  value  structure.  In  this  respect  they  provide 
more  information  for  comparison. 

First,  the  performance  of  each  statistic  will  be  analyzed  across 
tables  in  the  order  GSK,  Kullback,  and  then  Pearson,  then  the  statistics 
will  be  compared.  The  tables  are  arranged  in  order  of  increasing  cell 
size  for  each  statistic  and  nominal  level  combination,  with  two-way 
tables  preceding  three-way  tables  and  all  tables  grouped  for  each 
statls tic. 

The  performance  of  the  GSK  statistic  is  the  easiest  to  inter¬ 
pret.  As  previously  discussed,  the  basic  data  reflected  the  highly 
conservative  nature  of  this  statistic.  The  estimated  exact  signifi¬ 
cance  levels  were  considerably  below  the  nominal  levels  except  for 
equiprobable  vectors  at  large  sample  sizes.  Beginning  with  the  two- 
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way  tables  and  comparing  the  CEV  distributions  as  the  number  of  cells 
increases,  it  is  quickly  apparent  that  the  performance  of  the  GSK 
statistic  with  respect  to  small  expected  values  significantly  deteri¬ 
orates.  For  the  2x2  table  the  upper  CEV  distributions  include  some 
expected  values  less  than  five  for  the  .10  nominal  level  and  some  less 
than  six  for  the  .05  and  .01  nominal  levels.  For  the  2x3  table  only 
values  less  than  six  are  included  for  the  .10  level  and  values  less 
than  seven  for  the  .01  level.  For  the  2x4  table  no  values  less  than 
10  were  included  at  any  nominal  level.  CEV  distributions  were  not  pro¬ 
vided  for  the  2x5  table  since  no  N  were  found  less  than  the  maximum 

m 

sample  si2es  used.  Monte  Carlo  estimations  were  not  made  for  the  GSK 
statistic  in  the  3x3  table  due  to  its  obvious  deteriorating  performance 
and  the  relatively  extensive  computational  requirements. 

Considering  now  the  three-way  tables,  the  Monte  Carlo  estima¬ 
tions  were  made  for  the  GSK  statistic  in  the  2x2x2  table  under  both 
the  no  second-order  interaction  hypothesis  and  the  complete  inter¬ 
action  hypothesis.  Estimations  were  not  made  for  the  2x2x3  table  for 
the  same  reasons  noted  above  for  the  3x3  table.  Comparing  the  results 
when  moving  from  the  one  degree  of  freedom, no-second  order  interaction 
test  to  the  four  degree  of  freedom, complete  interaction  test,  the  per¬ 
formance  of  the  GSK  statistic  is  significantly  degraded  at  all  the 
nominal  levels  of  significance.  For  the  four  degree  of  freedom  test, 

only  the  equiorobable  vector  yielded  N  within  the  maximum  sample  sizes, 

m 

and  these  N  were  estimated  to  be  equal  to  these  maximum  values, 
m 

The  results  above  indicate  that  for  a  given  independence  test  the 
GSK  statistic  is  less  robust  with  respect  to  small  expected  values  as 


the  number  of  cells  Increases,  end  that  within  a  given  size  table  the 
statistic  Is  less  robust  as  the  degrees  of  freedom  of  the  test  in¬ 
creases.  Also,  the  statistic  Is  ultra-conservative  and  ineffective 
for  most  independence  hypothesis  testing  situations  under  the  log- 
linear  model. 

Results  for  the  Kullback  statistic  are  much  more  difficult  to 
interpret  and  do  present  some  inconsistencies.  First,  considering  the 
two-way  tables  at  the  .10  nominal  level,  from  the  2  *  2  to  the  2x3  table 
there  is  very  little  change  in  the  performance  of  the  statistic.  The 
minimum  expected  value  (MCE)  in  the  upper  part  of  the  CEV  distributions 
for  the  2x2  table  is  less  than  four,  and  there  are  four  separate 
vectors  that  have  that  small  a  value.  For  the  2x3  table  there  is  one 
vector  with  a  MCE  less  than  three  and  one  additional  vector  with  a  MCE 
less  than  four.  However,  there  are  two  vectors  in  the  lower  part  of 
the  2x3  table  (where  the  statistic  does  not  satisfy  the  criterion)  that 
have  CEV  distributions  with  MCE  just  less  than  four.  Here  there  is  an 
inconsistency,  where  some  vectors  indicate  improvement  in  performance 
and  others  indicate  a  degradation.  The  2x3  table  does  contain  a 
"richer"  CEV  structure  than  the  2x2  table  at  the  higher  ends  of  the 
upper  distributions,  but  this  may  be  offset  by  more  vectors  in  the  lower 
distributions  for  the  2x3  than  the  2x2  table.  Moving  to  the  2x4 
table,  the  distributions  Indicate  a  slightly  poorer  performance  for 
the  statistic.  For  the  3x3  and  2x5  cables  the  distributions  indi¬ 
cate  very  little  change.  Similar  results  are  obtained  at  the  .05  level. 

Results  at  the  .01  level  are  very  Inconsistent.  This  may  be 
caused  in  part  by  the  difficulty  in  making  some  N  determinations  for 
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Che  statistic  at  the  .01  level.  As  previously  mentioned  the  quadratic 

nature  of  the  Kullback  statistic  affected  the  N  determinations  at  the 

m 

extreme  vectors.  This  was  especially  critical  at  the  .01  nominal 
level.  For  the  most  extreme  vectors,  determinations  of  N  greater  than 

TO 

maximum  sample  sizes  were  relatively  easy  to  make.  For  the  mean  equi- 
probable  vectors* quadratic  effects  were  not  a  problem.  However*  for 
the  middle  vectors  it  was  difficult  to  determine  if  the  exact  signi¬ 
ficance  level  would  continue  within  the  confidence  interval  as  N  in¬ 
creased  beyond  the  maximum  N.  Therefore*  the  CSV  distributions  for  these 
vectors  appear  less  stable.  At  this  .01  level  the  lnconstencies  pre¬ 
sented  seem  to  indicate  neither  general  improvement  nor  degradation. 
Comparing  the  2x2  and  2x5  tables*  the  upper  CEV  distributions  are 
"richer"  for  the  2x5  cable  for  the  larger  expected  values*  but  for 
expected  values  less  than  five*  four,  etc.*  there  is  very  little  dif¬ 
ference.  The  lower  CEV  distributions  provide  no  additional  information. 

These  results  indicate  that  the  robustness  of  the  Kullback 
statistic  with  respect  to  small  expected  values  does  not  change  as  the 
number  of  cells  increase.  ' 

Now  considering  the  three-way  tables,  the  performance  of  the 
Kullback  statistic  slightly  improves  from  the  one  degree  of  freedom*  no 
second-order  interaction  test  to  the  four  degree  of  freedom*  complete 
Independence  test  in  the  2x2x2  table.  This  Improvement  is  more 
definite  for  smaller  nominal  levels.  Moving  to  the  seven  degree  of 
freedom,  complete  interaction  test  for  the  12  cell*  2x2x3  table* 
again  there  are  inconsistencies  which  indicate  no  apparent  trend  at 
any  of  the  nominal  levels.  These  results  agree  with  those  of  two-way 
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tables  with  respect  to  Increased  number  of  cells  for  a  given  hypothesis 
test.  In  comparing  the  different  hypothesis  tests  within  the  same  size 
table,  the  Kullback  statistic  performs  better  for  the  higher  degree  of 
freedom  test. 

The  results  for  the  Pearson  statistic  are  relatively  easy  to 
interpret  and  are  very  consistent.  Initially  considering  the  two-way 
tables,  as  the  number  of  cells  increase's,,  the  CEV  distributions  improve 
since  there  are  smaller  expected  values  and  more  small  expected  values 
where  the  statistic  satisfies  the  given  criterion.  This  is  true  at  all 
the  nominal  levels  of  significance.  More  specifically,  considering  the 
.10  nominal  level  CEV  distributions  for  the  2x2  table,  the  largest 
number  of  expected  values  less  than  one  is  two,  and  the  largest  number 
less  than  five  is  four.  For  the  2x3  table  these  numbers  are  three  and 
six,  respectively;  for  the  2x4  table,  four  and  eight;  for  the  3x3 
table,  five  and  nine;  and  for  the  2x5  table,  five  and  ten.  This 
improvement  indicates  that  the  Pearson  statistic  is  more  robust  with 
respect  to  small  expected  values  as  the  number  of  cells  increases. 

Now  considering  the  CEV  distributions  for  the  three-way  tables, 
the  performance  of  the  Pearson  statistic  improves  at  all  nominal 
levels  when  moving  from  the  one  degree  of  freedom, no  second-order  inter¬ 
action  test  to  the  four  degree  of  freedom, complete  independence  test 
in  the  2x2x2  table.  The  performance  further  improves  when  moving  to 
the  seven  degree  of  freedom, complete  independence  test  in  the  2x2x3 
table.  These  improvements  are  easily  distinguished  by  noting  the 
significant  Improvement  in  the  "richness"  of  CEV  structure.  These 
results  further  demonstrate  the  improved  robustness  of  the  Pearson 
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statistic  with  respect  to  small  expected  values  as  the  number  of  cells 
Increase,  and  they  indicate  an  improved  robustness  within  a  given  size 
table  for  higher  degree  of  freedom  tests. 

It  should  be  noted  that  comparisons  were  not  made  across  table 
dimensions.  It  does  not  seem  reasonable  to  compare  different  "type" 
tests  except  within  the  same  table  size  under  the  hierarchial  structure 
of  the  log- linear  model.  However,  comparisons  could  be  considered 
between  the  two-way  Independence  tests  (complete  independence  for  the 
two-way  table)  and  the  three-way  complete  independence  tests.  To  be 
consistent  these  comparisons  should  be  made  with  respect  to  the  number 
of  cells. 

First,  considering  the  eight  cell,  2x4  and  2x2x2  tables  and 
the  complete  independence  tests,  the  CEV  distributions  for  all  three 
statistics  at  all  three  nominal  levels  are  remarkably  similar,  without 
exception.  This  would  indicate  that  for  a  given  type  test  the  number 
of  cells  alone  (regardless  of  table  dimension)  determines  the  perform¬ 
ance  characteristics  for  each  statistic.  It  was  previously  shown  for 
each  statistic  that  results  for  the  two-way  and  three-way  tables  were 
consistent  with  respect  to  the  increase  in  number  of  cells.  Consider¬ 
ing  now  the  2x5  and  2x2x3  tables  with  10  and  12  cells,  respectively, 
the  results  are  again  consistent  for  all  three  statistics  at  all  three 
nominal  levels.  These  observations  indicate  that  within  a  "type" 
hypothesis  the  previous  results  can  be  extended  to  general  statements 
with  respect  to  the  number  of  cells  across  table  dimensions. 

These  CEV  distributions  provide  an  excellent  means  for  comparing 
the  performance  of  these  statistics  with  respect  to  small  expected 
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values.  Clearly,  the  Pearson  statistic  is  superior  for  all  the  two-way 
tables.  Its  exact  levels  of  significance  enter  the  95  percent  confi¬ 
dence  intervals  at  smaller  N  (N  )  than  the  levels  of  either  the  Kullback 

m 

or  GSK  statistic  for  every  two-way  table,  vector,  and  nominal  level 
combination.  Therefore,  the  CEV  distributions  for  the  Pearson  statistic 
are  "richer",  indicating  the  statistic  is  more  robust  with  respect  to 
small  expected  values.  For  the  two-way  tables  the  performance  of  the 
GSK  statistic  is  only  comparable  to  that  of  the  Kullback  statistic  in 
the  2x2  table  at  or  near  the  equiprobable  vector.  In  fact,  it  is 
slightly  superior  at  the  .10  and  .05  nominal  levels.  But, the  perform¬ 
ance  of  the  GSK  statistic  quickly  deteriorates  away  from  the  equiprob¬ 
able  vector  and  for  larger  tables.  In  these  cases  the  Kullback  statis¬ 
tic  is  clearly  superior. 

For  the  three-way  tables  the  GSK  statistic  shows  some  strength 
for  the  one  degree  of  freedom  test  of  no  second-order  interaction  in 
the  2x2x2  table.  Its  performance  is  slightly  better  than  those  of 
either  the  Kullback  or  Pearson  at  the  .10  level  and  comparable  at  the 
.05  and  .01  levels.  For  the  four  degree  of  freedom, complete  Interaction 
test  in  the  2x2x2  table,  the  GSK  statistic  performs  very  poorly,  the 
Kullback  statistic  shows  some  improvement  and  is  superior  to  the  GSK, 
and  the  Pearson  again  dominates  the  other  statistics.  In  the  2x2x3 
table  the  Pearson  statistic  is  clearly  superior  since  its  performance 
improves  from  the  2x2x2  table,  while  the  performance  of  the  Kullback 
statistic  shows  little  change, and  the  performance  of  the  GSK  statistic 
would  be  expected  to  deteriorate. 
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5.2,3  uz  and  Other  Parameters 

The  Individual  vectors  for  the  CEV  distributions  have  been 
arranged  in  order  of  decreasing  second  moment,  y2»  of  the  probability 
vectors,  y2  is  a  measure  of  the  spread  of  the  probability  vector  and 
is  calculated  as 


U2  -  !  P?/k  -  (1/k)2,  (5-1) 

i-1  1 

where  the  p^  are  the  cell  probabilities  and  k  is  the  number  of  cells. 

Interpreting  from  the  calculations  of  N  ,  the  data  in  Appendix 

m 

K  Indicate  that  the  statistics  generally  perform  better  as  the  y2  Para¬ 
meter  decreases.  The  best  performance  (smallest  N  )  is  near  the  equi- 

m 

probable  vector,  where  y2  *  0.  Several  authors  have  previously  suggested 
that  equiprobable  structures  are  best  for  the  use  of  the  Pearson  statis¬ 
tic  under  small  expected  values  [e.g.,  see  Roscoe  and  Byars  (1971)]. 

From  this  data  it  is  Interesting  to  observe  the  significant  relation¬ 
ship  between  N  and  y_  for  the  Kullback  and  GSK  statistics  and  to  some 
m  z 

extent  for  the  Pearson  statistic. 

The  y^  parameter  not  only  provides  an  indication  of  a  single 
small  probability  cell,  but  also  provides  a  measure  of  the  relative 
smallness  of  the  probabilities  of  all  cells.  The  independence  structure 
requires  that  a  number  of  small  probability  cells  be  balanced  with  a 
relatively  equal  number  of  large  probability  cells.  Although  there  are 
perturbations,  the  data  in  Appendix  K  generally  indicate  that  for  a 
given  table  these  statistics  perform  worse  both  as  the  smallest  expect- 


ed  value  decreases  and  as  Che  number  of  small  expected  values  Increases. 

Although  provides  a  convenient  parameter  for  each  Individual 
table,  comparisons  across  tables  Is  difficult.  Under  multinomial  sam¬ 
pling  as  the  number  of  cells  increases,  the  probability  structure  becomes 

"diluted".  For  the  two-way  tables  under  the  Independence  hypothesis 

2 

the  maximum  y^  approaches  1/k  -  1/k  .  Table  17  lists  these  maximum  y2 
for  the  two-way  tables  of  this  study.  For  a  given  sample  size  a  more 

Table  17.  Maximum  y2 :  Two-Way  Tables 

Table  k  max  y^ 

2  x  2  4  .1875 

2  x  3  6  .1389 

2x4  8  .1094 

3  x3  9  .0988 

2  x5  10  .0900 

diluted  structure  creates  more  small  expected  values,  thus  minimum  N 
calculations  are  not  directly  comparable. 

It  was  thought  that  a  correction  for  this  dilution  effect  might 
be  useful.  The  y2  values  were  "standardized"  for  each  table  by  dividing 
by  the  maximum  y2  values  of  Table  17, 

y’  -  y2/max  y2>  (5-2) 

An  attempt  was  then  made  to  compare  the  minimum  N  determinations  for 
these  standardized  y2  across  tables.  These  comparisons  were  highly 


erratic 


Another  parameter  was  considered  that  would  account  for  the 

increase  in  table  size.  A  coefficient  of  variation  (V)  was  calculated 

using  the  ratio  of  the  standard  deviation  of  the  expected  values  and 

the  mean  of  the  expected  values.  The  variance  of  the  expected  values 
2 

is  N  v*2»  30  Chat, 

V  -  N(y2)1/2/(N/k)  -  k(u2)1/2.  (5-3) 

Calculations  were  made  and  comparisons  attempted  across  tables.  Again 
the  comparisons  were  very  erratic. 

Other  parameters  were  considered,  including  the  third  and  fourth 
moments  of  the  probability  structures  and  coefficients  of  skewness  and 
kurtossis.  Linear  models  with  combinations  of  these  parameters  were 
also  considered.  None  of  these  parameters  or  models  provided  consistent 
results.  u2  was  the  best  measure  for  performance  within  tables,  but 
the  effects  across  tables  did  not  appear  sufficiently  stationary  to 
model  effectively.  The  CEV  distributions  provided  the  only  reasonable 
vehicle  to  compare  results  across  tables. 
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CHAPTER  VI 

CONCLUSIONS  AND  RECOMMENDATIONS 

As  stated  in  the  Introduction,  the  principal  focus  of  this 
research  was  the  investigation  of  current  methodologies  for  the  analy¬ 
sis  of  contingency  tables  with  respect  to  the  classic  problem  of  small 
expected  values.  The  more  specific  purpose  of  this  research  was  to 
investigate  the  robustness  characteristics  of  the  associated  statistics 
with  respect  to  small  expected  values  as  the  size  of  the  table  Increases. 
A  secondary  purpose  was  to  compare  these  statistics  rad  make  recommen¬ 
dations  for  their  u*e.  With  these  considerations  this  chapter  will 
present  the  conclusions  of  tibia  investigation,  provide  recommendations 
for  the  use  of  the  statistics  investigated,  and  make  recommendations 
for  future  research. 

6.1  Conclusions 

From  the  results  of  this  study  the  following  conclusions  can  be 

drawn: 

1)  The  basic  data  (Appendices  F  and  H)  reflect  the  asymptotic 
nature  of  these  statistics.  Generally,  as  the  sample  size  increases,  the 
exact  levels  of  the  statistics  approach  the  nominal  levels,  indicating 
that  the  exact  distributions  approach  the  chi- squared  distribution. 

2)  The  minimum  and  maximum  significance  levels  for  minimum  cell 
expectation  intervals  (MCEI)  (Appendix  I)  demonstrate  that  tha  perform¬ 
ance  of  these  statistics  within  a  certain  size  table  generally  improves 


as  the  MCE  increases.  These  levels  do  not  provide  reasonable  compari¬ 
sons  across  tables. 

3)  The  second  moment  of  the  underlying  probability  vector,  p 
calculated  from  (5-1),  is  a  convenient  parameter  for  measuring  the 
performance  of  a  statistic  within  a  certain  size  table.  The  para¬ 
meter  is  a  measure  of  how  "extreme"  the  underlying  probability 
structure  is  with  respect  to  the  equiprobable  vector, where  \i^  *  0.  The 
^2  parameter  not  only  accounts  for  a  single  small  cell  probability  but 
also  a  number  of  small  cell  probabilities.  In  general,  as  y_  decreases, 
the  performance  of  these  statistics  improves.  Each  statistic  performs 
best  at  or  near  the  equiprobable  vector. 

4)  The  determination  of  minimum  sample  sizes  (Nffl)  provides  the 
parameters  for  calculating  critical  expected  value  (CEV)  distributions. 
These  CEV  distributions  provide  a  vehicle,  not  only  for  comparing  the 
performances  of  these  statistics,  but  also  for  determining  trends  of 
the  robustness  of  these  statistics  with  respect  to  small  expected  values 
as  the  size  of  the  table  increases. 

5)  The  GSK  statistic  is  highly  conservative  to  the  point  of  being 
ineffective  as  a  test  statistic  for  most  of  the  hypothesis  situations 

in  this  study,  particularly  at  the  more  extreme  probability  vectors. 

The  statistic  appears  to  be  significantly  biased  by  the  zero  cell  cor¬ 
rection  procedure,  particularly  at  the  more  extreme  probability 
vectors  where  the  probability  of  obtaining  a  sampling  zero  is  rela¬ 
tively  high.  The  performance  of  the  GSK  statistic  significantly 
deteriorates  as  the  number  of  cells  of  the  table  increases  even  for 
the  more  equiprobable  structures.  The  CEV  distributions  indicate  that 


the  statistic  is  less  robust  with  respect  to  small  expected  values  as 

the  size  of  the  table  increases. 

6)  The  Kullback  statistic  generally  tends  to  be  liberal  and 

demonstrates  a  "quadratic"  asymptotic  nature,  especially  at  the  more 
extreme  probability  vectors.  The  statistic  does  not  perform  nearly  as 
well  as  the  Pearson  but  does  generally  perform  better  than  the  GSK.  The 
Kullback  statistic  performs  best  at  the  lowest  nominal  level,  a  *  .01. 
With  respect  to  an  increase  in  the  number  of  cells,  the  performance  of 
the  statistic  is  inconsistent.  In  general,  the  Kullback  statistic 
demonstrates  little  change  in  robustness  with  respect  to  small  expected 
values  as  the  size  of  the  table  increases. 

7)  Of  the  three  statistics  evaluated,  the  Pearson  statistic  is 
clearly  superior  in  performance.  In  many  cases  it  tends  to  be  conserva¬ 
tive,  but  its  exact  levels  of  significance  approach  the  nominal  levels 
relatively  quickly  as  the  sample  size  increases.  The  performance  of  the 
statistic  significantly  improves  as  the  number  of  cells  increases.  The 
P ’arson  statistic  is  more  robust  with  respect  to  small  expected  values 
as  the  size  of  the  table  Increases. 

8)  The  statistics  were  evaluated  for  the  2x2x2  table  for  both 
the  one  degree  of  freedom,  no  second-order  interaction  hypothesis  and 
the  four  degree  of  freedom,  complete  interaction  hypothesis.  From  the 
one  degree  of  freedom  to  the  four  degree  of  freedom  test,  the  perform¬ 
ance  of  the  Pearson  statistic  significanctly  improves,  that  of  the 
Kullback  statistic  slightly  improves,  and  that  of  the  GSK  statistic 
deteriorates.  These  results  indicate  that,  at  least  for  hierarchical 
hypotheses,  the  Pearson  and  Kullback  statistics  are  more  robust  and 
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the  GSK  statistic  is  less  robust  with  respect  to  small  expected  values 

as  the  number  of  degrees  of  freedom  of  the  test  increases. 

From  these  conclusions  the  following  recommendations  are  made: 

1)  Unless  a  highly  conservative  test  statistic  is  desired,  the 
GSK  statistic  should  not  be  used  in  hypothesis  testing  under  the  log- 
linear  model  structure.  This  recommendation  is  most  important  when  the 
observed  structure  contains  sampling  zero  cells,  where  some  technique 
would  be  needed  to  correct  for  these  cells. 

2)  The  Pearson  statistic  should  be  preferred  over  the  Kullback 
in  these  independence  hypothesis  testing  situations.  This  is  especially 
true  for  larger  contingency  tables. 

3)  The  analyst  should  feel  confident  in  the  use  of  the  Pearson 
statistic  for  large  tables  even  when  some  expected  values  are  very 
small. 


6.2  Recommendations  for  Future  Research 

This  study  provided  the  first  broad  investigation  of  the  robust¬ 
ness  of  these  statistics  with  respect  to  small  expected  values  as  the 
size  of  the  table  increases.  However,  this  study  was  limited  to  certain 
specific  conditions  including  the  tables,  hypotheses,  sampling  model, 
and  log-linear  model.  Recommendations  for  future  research  include: 

1)  Investigations  of  the  robustness  of  these  statistics  with 
respect  to  other  hypothesis  tests  and  sampling  models;  in  particular, 
the  homogeneity  hypothesis  under  product-multinomial  sampling. 

2)  Investigation  of  the  robustness  of  statistics  using  other 
underlying  models  and  formations  of  interaction  hypotheses;  in  parti¬ 
cular,  the  performance  of  the  GSK  statistic  using  the  linear  model  and 


corresponding  independence  hypotheses. 

3)  A  further  investigation  of  the  relationship  of  or 
associated  parameters,  and  the  performance  of  these  statistics. 

4)  Investigation  of  the  effect  that  "closeness"  of  the  exact 
levels  of  significance  to  the  nominal  levels  has  on  the  power  of  the 
test  under  selected  alternative  hypotheses. 

5)  Investigation  of  the  robustness  of  the  GSK  statistic  using 
other  methods  for  correction  of  sampling  zero  cells;  in  particular,  the 
.05  correction  of  Goodman  (1970)  or  the  Bhapkar  (1979)  correction. 


APPENDIX  A 


APPROXIMATIONS  TO  MDI  STATISTICS 

This  appendix  follows  the  derivations  given  in  Gokhale  and 
Kullback  (1978,  Appendix)  except  for  the  Maclaurin  series  expansion 
of  (A-4). 

Let  T^ui)  (i-1,2 . n)  be  a  set  of  linearly  independent 

statistics  defined  over  the  set  of  cells  ft.  The  MDI  theorem  proved 
by  Kullback  (1959)  gives  the  value  of  p(o>)  which  minimizes  the  dis¬ 
crimination  information, 

I(p:ir)  -  l  p(w)  in(p(w)  /tt(oj)  ) ,  (A-l) 

ft 

between  two  distributions  £  and  ir  over  the  family,  P  of  ^-distributions 
which  satisfy 

l  T. (w)p(w)  -  0*;  i-l,2,...,n,  (A-2) 

ft  1  1 

for  given  9*.  If  ir(cj)  satisfies  (A-2),  then  I(£:jr)  »  0.  Otherwise, 
the  MDI  theorem  gives 

n 

p*(w)  -  exp(  £  r.T  (u))ir(a))/M(T), 

i  *  * 


(A-3) 


\dr. 


where 


M(t)  -  l  exp(  l  T  T  (w))7r((jj), 
Q  i-1  1  1 


(A-4) 


The  T  parameters  are  equivalent  to  Lagrangian  multipliers  whose  values 
are  defined  in  terms  of  the  known  0*, 


9*  -  M<T) 


l  [exp(  l  t  Ti(u)))T1((u)iT((jj)]/M(T) ; 
£2  i-1 

i*l|2j • • • ,n. 


(A-5) 


Note  that  when  -  Tj  •  ***  *  Tq  •  0,  (A-3)  implies  that 


p*(cu)  -  e^irCw)/^  e®»ir(w)  ■  tt(w). 

n 


(A-6) 


From  (A-3)  the  familiar  log-linear  model  of  Chapter  III  can  be  written. 


£n[p*(u>)/iT(u)  ]  -  L  +  l  t.T  (w), 

i-1  x  1 


(A-7) 


where  L  -  -Jin  MOt)  .  The  minimum  value  of  the  discrimination  Information 
in  (A-l)  is  then 


■  ”  •  "  » ,*  *  *  «  '  *  '  * 
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I(p*:ir)  -  £  p*(w)2n(p*(ui)/ff(u))) 
ft 

n  n 

-  I  ([exp(  l  T  T  (u))ir(u)/M(T)][L  +  £  t.T.(w)]} 

ft  1-1  1  1  i-1  1  1 


a  n 

•  I  T  (  l  [exp(  l  T  T  (u)))T.(w)Tr(aj)/M(T)} 
1-1  1  ft  1-1  1  1  X 


+  (L/M(t))  l  exp(  l  TT(w)  )*(<*>), 
ft  1-1  1  t 


and  substituting  from  (A-4)  and  (A- 5) , 


I(L* :ir )  -  l  t  9  *  +  (L/M(t )  )M(x  ) 

1-1  1  1 

n 

-  I  T  9*  +  L.  (A-8) 

1-1  1  1 


From  Kullback  (1959,  1970)  Che  following  duality  relations  exist 
between  the  9  and  T  parameters  and  the  covariance  matrix  of  the  . 

Letting 


(d6*)f  =  (d9*  d9* . d9*) 

“ “  l  i  n 


(dr)'  -(dTlfdT2,...,dTn), 


and 


d0*  -  £*dr,  dr  -  ^*"1d0. 


where  2*  is  the  covariance  matrix  of  the  T^(u)  with  respect  to  the 
p*(w)  distribution,  and 


T  -  sjj  -  l  (^(w)  -  0*)(T*(w)  -  6^)71(00) 


s 


3t. 

Q*  —  -i.4 

n  *  * 

an 


From  (A-4)  a  Maclaurin  series  expansion  can  be  used  to  derive 

an  approximation  for  In  M(x)  up  to  quadratic  terms.  Using 
x  2 

e  2l  +  x  +  x/2!  and  substituting  into  (A-4) , 


M(T)  =  l  [1  +  2  T  T .(«)  t  T  (m))2]it(u) 

ft  ill  i  ±  11 

-  I  ir(u)  +  l  T.  I  T.  (u>)tt(uj) 
ft  i  1  ft  X 

+  7  I  ir(w)(2  T.T.(u)))2. 

^  ft  i  1  1 


Letting 


0.-2  T.(w)tt(w), 

1  ft 
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noting  that  £  Tr(o>)  ■  1,  and  expanding  the  third  term, 
ft 

M(T)  3  1  +  J  T  0  +-|  l  TT(W)E  l  T  T  T  (W)T  (W), 

i11  Zft  ij  131  j 


Now,  using  £n(l  +  x)  :  x  -  and  disregarding  terms  larger  than 

quadratic. 


M(T)  *  l  T  0  +±l  IT  (to)  l  l  T  T  T  (UJ)T.(W)  -  kl  T.e.y 

i  z  ft  i  1  1J  1  3  zi  11 


Looking  at  the  third  term  and  expanding, 


(i Tl9l>2 '  I  j  t‘tj9i9j 


Adding  and  subtracting  8^9  , 


«  Ti9i>2  - 1  \  Vj'Vj +  9i9j  -  W' 


Using  Equation  (A- 11) ,  substituting 


6.  -  l  T  (u)Tr(w) 
3  ft  3 


in  term  one  and 


*  ••*?■**  m  *  •  * 
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3  £n  M(x) 


i*l  #  i  •  |ii« 


The  matrix  £  ■  (ct^)  is  the  covariance  matrix  of  the  T^w)  with 
to  the  ir((o)  distribution. 

Relations  for  2I(£*:tt)  can  now  be  derived  using  £  and  £~ 
(A-12),  (A-13),  and  (A- 14)  in  (A-8) : 


2I(2*:tt)  -  2(1 
i 

*  2(1 

i 

-  2(1 

i 

-n 

i  1 


Ti0j  -  In  M(T)) 

Ti(9i  +  |  °«V  -  <*  Ti9i  +  5  l  |  Wij» 

J  °yTiTj  ‘  2 1  TiV 

aijTiTj  * 


In  matrix  notation 


2I(£*:7r)  «  _t'  X  t. 

In  matrix  notation  Equation  (A-14)  can  be  written 

9_*  z  _9  +  l  t_* 


(A-14) 


respect 

and 


(A-15) 


Solving  for  _r. 
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I  =  J_1(e*  -  e) . 


Substituting  into  (A-15), 


2i(£*:jt)  s  c_9*  -  e^r1!  r\i*  ~  £> 


<1*  - - 1). 


(A-16) 


Now,  the  set  of  T^w)  (i*l,2,. . . ,n)  functions  can  be  partitioned 
into  two  sets:  one  set,  H^,  where  from  (A-2)  and  (A-ll) 


9*  »  9i;  i"l,2 . nA; 


(A-17) 


and  the  remaining  n-nA  T^Cou)  into  the  set  H^.  The  other  matrices  can 
be  partitioned  correspondingly: 


0*’  -  <0j',  0J'),  0’  -  (9|,  0^),  T*  -  (T^,  Tj), 


(A-18) 


and  the  covariance  matrix. 


AAA  AAB 


ABA  ABB 


(A-19) 


Where  IaA  18  nA  *  V  IbB  is  X  (n-nA)  •  Snd  (lBA)f  "  IaB 

is  n.  x  (n-n  ) . 

A  & 


Substituting  these  relations  into  (A- 14),  _9^  »  _9  +  £  _T, 


^  =  4+Iaa4  +  IaiIb 

ifi  =  -^b  +  Abb-b  +  Aba  1a- 


But  the  partition  (A-18)  was  established  based  on  (A-17),  so 

_9*  «  6^,  and  the  approximations  (A- 15)  and  (A-16)  can  be  written  in 

A  A 

terms  of  the  partition.  From  (A-20) 


Y  aaT..  +  L„  Tn  *  0^  -  9  «  0; 
j“  AA— A  J^AB—B  —A  —A  — 


therefore , 


~A  *  IaaIaB^B' 


From  (A-15)  and  the  partition, 


2K£*:77)  s 


La  I 


£AA  iiABl  I  —A 


I  I  T 

"BA  ~BB  -B 


*  i;La  -a  +  ^bIba-a  +  -aIab-b  +  -bIbb-b 

■  -A(1aA^A  +IaB^B)  +  — B  BA  —A  +  IbB-^ 

"  Ia^  +  -^b^Ibb-b  “  IbaIaa  Iab^b) 


Substituting  into  (A-21), 


f  f, 


NO. 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

13 

14 

15 

16 

17 

18 

19 

20 
21 


TABLE:  2x3 


pl*  P-1  p*  2 

.1  .1  .1 

.1  .1  .3 

.1  .2  .2 

.1  .2  .4 

.2  .1  .2 

.2  .1  .4 

.2  .2  .3 

.2  .3  .3 

.3  .]  .] 

.3  .1  .3 

.3  .2  .2 

.3  .2  .4 

.4  .1  .2 

.4  .1  .4 

.4  .2  .3 

.4  .3  .3 

.5  .1  .1 

.5  .1  .3 

.5  .2  .2 

.5  .2  .4 

.5  1/3  1/3 


(P11*P12,P13,P21,P22’P23) 

(.01, 

.01, 

.08, 

.09, 

.09, 

.72) 

(.01, 

.03, 

.06, 

.09, 

.27, 

.54) 

(.02, 

.02, 

.06, 

.18, 

.18, 

.54) 

(.02, 

.04, 

.04, 

.18, 

.36, 

.36) 

(.02, 

.04, 

•  14, 

.08, 

.16, 

.56) 

(.02, 

.08, 

.10, 

.08, 

•  32, 

.40) 

(.04, 

.06, 

.10, 

.16, 

.24, 

.40) 

(.06, 

.06, 

.08, 

.24, 

.24, 

.32) 

(.03, 

.03, 

•  24, 

.07, 

.07, 

.56) 

(.03, 

•  09, 

.18, 

.07, 

.21, 

.42) 

(.06, 

.06, 

.18, 

•14, 

.14, 

.42) 

(.06, 

.12, 

.12, 

•  14, 

.28, 

.28) 

(.04, 

.08, 

.28, 

.06, 

.12, 

•42) 

(.04, 

.16, 

.20, 

.06, 

•  24, 

.30) 

(.08, 

.12, 

.20, 

.12, 

.18, 

.30) 

(.12, 

•  12, 

.16, 

.18, 

.18, 

.24) 

(.05, 

.05, 

.40, 

.05, 

.05, 

.40) 

(.05, 

.15, 

.30, 

.05, 

.15, 

.30) 

(.10, 

.10, 

.30, 

.10, 

.10, 

.30) 

(.10, 

.20, 

.20, 

.10, 

.20, 

.20) 

(1/6, 

1/6, 

1/6, 

1/6, 

1/6, 

1/6) 

w«  *  *  v  ■'  *  v»  i'-  7 


■* 


k> 


i 
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NO. 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

13 

14 

15 

16 

17 

18 

19 

20 
21 


TABLE:  2  x4 


Pl’  P-l  p-2 

.1  .1  .1 

.1  .1  .1 

.1  .1  .2 

.1  .2  .2 

.2  .1  .1 

.2  .1  .1 

.2  .1  .2 

.3  .2  .2 

.3  .1  .1 

.3  .1  .1 

.3  .1  .2 

.3  .2  .2 

.4  .1  .1 

.4  .1  .1 

.4  .1  .2 

.4  .2  .2 

.5  .1  .1 

.5  .1  .1 

.5  .1  .2 

.5  .2  .2 

.5  1/4  1/4 


pl- 

p2* 

1 


P-3  (pll,Pl2,Pl3,Pl4,p21,p22,P23.P24) 


.1 

(-01, 

.01, 

.01, 

.07, 

.09, 

.09, 

.09, 

.63) 

.3 

(-01, 

.01, 

.03, 

.05, 

.09, 

.09, 

.27, 

.45) 

.2 

(-01, 

.02, 

.02, 

.05, 

.09, 

.18, 

.18, 

.45) 

.2 

(-02, 

.02, 

.02, 

.04, 

.18, 

.18, 

.18, 

.36) 

.2 

(.02, 

.02, 

.04, 

.12, 

.08, 

.08, 

.16, 

.48) 

.4 

(.02, 

.02, 

.08, 

.08, 

.08, 

.08, 

.32, 

.32) 

.3 

(.02, 

.04, 

.06, 

.08, 

.08, 

.16, 

•  24, 

.32) 

.3 

(.04, 

.04, 

.06, 

.06, 

.16, 

.16, 

.24, 

.24) 

.1 

(.03, 

.03, 

.03, 

.21, 

.07, 

.07, 

.07, 

.49) 

.3 

(.03, 

.03, 

.09, 

.15, 

.07, 

.07, 

.21, 

•  35) 

.2 

(.03, 

.06, 

.06, 

.15, 

.07, 

.14, 

.14, 

.35) 

.2 

(-06, 

.06, 

.06, 

.12, 

.14, 

•  14, 

.14, 

.28) 

.2 

(.04, 

.04, 

.08, 

.24, 

.06, 

.06, 

.12, 

.36) 

.4 

(-04, 

.04, 

.16, 

.16, 

.06, 

.06, 

.24, 

.24) 

.3 

(-04, 

.08, 

•12, 

.16, 

.06, 

.12, 

.18, 

.24) 

.3 

(.08, 

.08, 

.12, 

.12, 

.12, 

•  12, 

.18, 

.18) 

.1 

(.05, 

.05, 

.05, 

.35, 

.05, 

.05, 

.05, 

.35) 

.3 

(.05, 

.05, 

.15, 

•  25, 

.05, 

.05, 

.15, 

.25) 

.2 

(.05, 

.10, 

.10, 

.25, 

.05, 

.10, 

.10, 

.25) 

.2 

(.10, 

.10, 

.10, 

.20, 

.10, 

.10, 

.10, 

.20) 

1/4 

(1/8, 

1/8, 

1/8, 

1/8, 

1/8, 

1/8, 

1/8, 

1/8) 

P11 

p12 

P13 

P14 

P21 

p22 

P23 

P24 

P-l  P-2  P-3  P-4 


N 


TABLE:  3x3 


Pll 

P12 

p13 

p2i 

p22 

p23 

P31 

p32 

p33 

P,1  P.2  P*3  1 


pl-  P2-  P*1  P-2 


.1  .1 
.1  .1 
.1  .1 
.1  .1 
.1  .2 
.1  .2 
.1  .2 
.1  .2 
.1  .3 

.1  .3 

.1  .3 

.1  .4 

.1  .4 

.1  .4 

.2  .2 
.2  .2 
.2  .3 

.2  .3 

.2  .4 

.3  .3 

1/3  1/3 


.1  .1 

.1  .3 

.2  .2 

.2  .4 

.1  .2 

.1  .4 

.2  .3 

.3  .3 

.1  .3 

.2  .2 

.2  .4 

.1  .4 

.2  .3 

.3  .3 

.2  .2 

.2  .4 

.2  .3 

.3  .3 

.2  .4 

.3  .3 

1/3  1/3 


(pirp12»p13* 

P21,P22,P23*P31,P32*P33^ 

(.01, 

.01, 

.08, 

.01, 

.01, 

.08, 

.08, 

.08, 

.64) 

(.01, 

.03, 

.06, 

.01, 

.03, 

.06, 

.08, 

.24, 

.48) 

(.02, 

.02, 

.06, 

.02, 

.02, 

.06, 

.16, 

.16, 

.48) 

(.02, 

.04, 

.04, 

.02, 

.04, 

.04, 

.16, 

.32, 

.32) 

(.01, 

.02, 

.07, 

.02, 

.04, 

•  14, 

.07, 

.14, 

.49) 

(.01, 

.04, 

.05, 

:02, 

.08, 

.10, 

.07, 

.28, 

.35) 

(.02, 

.03, 

.05, 

.04, 

.06, 

.10, 

•  14, 

•  2], 

.35) 

(.03, 

.03, 

.04, 

.06, 

.06, 

.08, 

.21, 

•  21, 

.28) 

(.01, 

.03, 

.06, 

.03, 

.09, 

.18, 

.06, 

.18, 

.36) 

(.02, 

.02, 

.06, 

.06, 

.06, 

.18, 

.12, 

.12, 

.36) 

(.02, 

.04, 

.04, 

.06, 

.12, 

.12, 

.12, 

.24, 

.24) 

(.01, 

.04, 

.05, 

.04, 

.16, 

.20, 

.05, 

.20, 

.25) 

(.02, 

.03, 

.05, 

.08, 

.12, 

.20, 

.10, 

.15, 

•  25) 

(.03, 

.03, 

.04, 

.12, 

•  12, 

.16, 

.15, 

.15, 

.20) 

(.04, 

.04, 

.12, 

.04, 

.04, 

.12, 

.12, 

.12, 

.36) 

(.04, 

.08, 

.08, 

.04, 

.08, 

.08, 

.12, 

.24, 

.24) 

(.04, 

.06, 

.10, 

.06, 

.09, 

.15, 

.10, 

.15, 

.25) 

(.06, 

.06, 

.08, 

.09, 

.09, 

.12, 

.15, 

.15, 

.20) 

(.04, 

.08, 

.08, 

.08, 

.16, 

.16, 

.08, 

.16, 

.16) 

(.09, 

.09, 

.12, 

.09, 

.09, 

.12, 

.12, 

.12, 

.16) 

(1/9, 

1/9, 

1/9, 

1/9, 

1/9, 

1/9, 

1/9, 

1/9, 

1/9) 

PL-  *.*,*_, 
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*11 

*12 

*13 

*14 

*15 

*1- 

*21 

*22 

*23 

*24 

*25 

*2* 

*•1 

*•2 

*•3 

*.4 

*•5 

1 

4  <pll'p 

12**13**14 

**15’ 

*21 ’*22 ’*23 **24 **25^ 

.1  (.01,. 01,. 01,. 01,. 06,. 09,. 09,. 09,. 09,. 54) 
.3  (.01,. 01,. 01,. 03,. 04,. 09,. 09,. 09,. 27,. 36) 
.3  (.01,. 01,. 02,. 03,. 03,. 09,. 09,. 18,. 27,. 27) 
.2  (.02,. 02,. 02,. 04,. 10,. 08,. 08,. 08,. 16,. 40) 
.2  (.02,. 02,. 04,. 04,. 08,. 08,. 08,. 16,. 16,. 32) 
.2  (.02,. 04, .04,. 04,. 06,. 08,. 16,. 16,. 16,. 24) 
.1  (.03,. 03,. 03,. 03,. 18,. 07,. 07,. 07,. 07,. 42) 
.3  (.03,. 03,. 03,. 09,. 12,. 07,. 07,. 07,. 21,. 28) 
.3  (.03,. 03,. 06,. 09,. 09,. 07,. 07,. 14,. 21,. 21) 
.2  (.04,. 04,. 04,. 08,. 20,. 06,. 06,. 06,. 12,. 30) 
.2  (.04,. 04,. 08,. 08,. 16,. 06,. 06,. 12,. 12,. 24) 
.2  (.04,. 08,. 08,. 08,. 12,. 06,. 12,. 12,. 12,. 18) 
.1  (.05,. 05,. 05,. 05,. 30,. 05,. 05,. 05,. 05,. 30) 
.3  (.05,. 05,. 05,. 15,. 20,. 05,. 05,. 05,. 15,. 20) 
.3  (.05,. 05,. 10,. 15,. 15,. 05,. 05,. 10,. 15,. 15) 
.2  (.10,. 10,. 10,. 10,. 10,. 10,. 10,. 10,. 10,. 10) 


TABLE  :  2x2x2 


pll- 
P12- 

Pl-1  pl*2  pl-- 


plll 

p112 

p121 

P122 

p21- 
P22* 

p2-l  p2-2  p2* • 


p211 

p212 

P221 

P222 

P-*l“  plll+p121+p211+p221 
p*  *2*  P112+P122+P212+P222 

N°‘  Pl-  P‘l-  P--l  (plll,p112,p121,p122,p211,p212*p221,p222) 


*1 

*2 

*3 

*4 

5 

6 

7 

8 
9 


.1  .1  .1 

.1  .1  .3 

.1  .1  .5 

.1  .2  .2 

.1  .2  .4 

.1  .3  .3 

.1  .3  .5 

.1  .4  .4 

.1  .5  .5 


(.001, .009, .009, .081, .009, .081, .081, .729) 
( . 003 , . 007 , . 027 , . 063 , . 027 , . 063 , . 243 , . 567) 
(.005,. 005,. 045,. 045,. 045,. 045,. 405,. 405) 
( . 004, . 016 , . 016 , . 064 , . 036 , . 144 , . 144 , . 576) 
(.008,. 012,. 032,. 048,. 072,. 108,. 288,. 432) 
(.009, .021,. 021, .049, .081, .189, .189,. 441) 
(.015,. 015,. 035,. 035,. 135,. 135,. 315,. 315) 
(.016,. 024,. 024,. 036,. 144,. 216,. 216,. 324) 
(.025,. 025,. 025,. 025,. 225,. 225,. 225,. 225) 


*10  .2  .2  .2 

11  .2  .2  .4 

12  .2  .3  .3 

13  .2  .3  .5 

14  .2  .4  .4 

15  .2  .5  .5 

16  .3  .3  .3 

17  .3  .3  .5 

18  .3  .4  .4 

19  .3  .5  .5 

20  .4  .4  .4 

21  .4  .5  .5 

22  .5  .5  .5 


(.008, 

.032, 

.032, 

.128, 

.032, 

.128, 

.128, 

.512) 

(.016, 

.024, 

.064, 

.096, 

.064, 

.096, 

.256, 

.384) 

(.018, 

.042, 

.042, 

.098, 

.072, 

.168, 

.168, 

.392) 

(.030, 

.030, 

.070, 

.070, 

.120, 

.120, 

.280, 

.280) 

(.032, 

.048, 

.048, 

.072, 

.128, 

.192, 

.192, 

.288) 

(.050, 

.050, 

.050, 

.050, 

.200, 

.200, 

.200, 

.200) 

(.027, 

.063, 

.063, 

.147, 

.063, 

.147, 

.147, 

.343) 

(.045, 

.045, 

.105, 

.105, 

.105, 

.105, 

.245, 

.245) 

(.048, 

.072, 

.072, 

.108, 

.112, 

.168, 

.168, 

.252) 

(.075, 

.075, 

.075, 

.075, 

.175, 

.175, 

.175, 

.175) 

(.064, 

.096, 

.096, 

.144, 

.096, 

.144, 

.144, 

.216) 

(.100, 

.100, 

.100, 

.100, 

.150, 

.150, 

.150, 

.150) 

(.125, 

.125, 

.125, 

.125, 

.125, 

.125, 

.125, 

.125) 

for 


"no  second-order'  interaction  hypothesis 


*  Not  used 


TABLE:  2><2x3 


P121  P 122  p123l  p12- 
pl*l  Pl*2  pl*  3  pl** 


p2 1 1  p2 1 2  P213 

P221  P222  P223 

P2* 1  P2*2  P2*  3 


NO.  Pl. 


p**l"  Plll+P121+P211+P221 
p*  *2*  P112+P122+P212+P222 
p**3*  p113+p123+p213+p223 

(plll,p112,p113’p12rp122,p123 
P-l*  P**l  P*  * 2  p2irp212,p213’p221’p222,p223 


(.001, 

.001, 

.008, 

.009, 

.009, 

.072, 

.009, 

.009, 

.072, 

.081, 

.081, 

.648) 

(.002, 

.002, 

.006, 

.018, 

.018, 

.054, 

.018, 

.018, 

.054, 

.162, 

.162, 

.486) 

(.002, 

.004, 

.014, 

.008, 

.016, 

.056, 

.018, 

.036, 

.126, 

.072, 

.144, 

.504) 

(.004, 

.006, 

.010, 

.016, 

.024, 

.040, 

.036, 

.054, 

.090, 

.144, 

.216, 

.360) 

(.003, 

.009, 

.018, 

.007, 

.021, 

.042, 

.027, 

.081, 

.162, 

.063, 

.189, 

.378) 

(.006, 

.012, 

.012, 

.014, 

.028, 

.028, 

.054, 

.108, 

.108, 

.126, 

.252, 

.252) 

(.004, 

.016, 

.020, 

.006, 

.024, 

.030, 

.036, 

.144, 

.180, 

.054, 

.216, 

.270) 

(.012, 

.012, 

.016, 

.018, 

.018, 

.024, 

.10-, 

.108, 

.144, 

.162, 

.162, 

.216) 

(.005, 

.005, 

.040, 

.005, 

.005, 

.040, 

.045, 

.045, 

.360, 

.045, 

.045, 

.360) 

(.010, 

.010, 

.030, 

.010, 

.010, 

.030 

.090, 

.090, 

.270, 

.090, 

.090, 

.270) 

(.004, 

.008, 

.028, 

.016, 

.032, 

.112, 

.016, 

.032, 

.112, 

.064, 

.128, 

.448) 

(.008, 

.012, 

.020, 

.032, 

.048, 

.080, 

.032, 

.048, 

.080, 

.128, 

.192, 

.320) 

(.006, 

.018, 

.036, 

.014, 

.042, 

.084, 

.024, 

.072, 

.144, 

.056, 

.168, 

.336) 

(.012, 

.024, 

.024, 

.028, 

.056, 

.056. 

.048, 

.096, 

.096, 

.112, 

.224, 

.224) 

(.008, 

.032, 

.040, 

.012, 

.048, 

.060, 

.032, 

.128, 

.160, 

.048, 

.192, 

.240) 

(.024, 

.024, 

.032, 

.036, 

.036, 

.048, 

.096, 

.096, 

.128, 

.144, 

.144, 

.192) 

(.010, 

.010, 

.080, 

.010, 

.010, 

.080, 

.040, 

.040, 

.320, 

.040, 

.040, 

.320) 

(.020, 

.020, 

.060, 

.020, 

.020, 

.060, 

.080, 

.080, 

.240, 

.080, 

.080, 

.240) 

rid* 


TABLE:  2x2x3  continued 


NO. 

Pi-- 

P-l- 

P--1 

P..2 

^plll ,p112'Pll3,p121 ,p122,p123x 
P2irP212’P213’P22rP222’P223; 

19 

.3 

.3 

.1 

.2 

(.009,. 018,. 063,. 021,. 042,. 147, 
.021,. 042,. 147,. 049,. 098,. 343) 

20 

.3 

.3 

.2 

.3 

(.018,. 027,. 045,. 042,. 063,. 105, 
.042,. 063,. 105,. 098,. 147,. 245) 

21 

.3 

.4 

.1 

.3 

(.012,. 036,. 072,. 018,. 054,. 108, 
.028,. 084,. 168,. 042,. 126,. 252) 

22 

.3 

.4 

.2 

.4 

(.024,. 048,. 048,. 036,. 072,. 072, 
.056,. 112,. 112,. 084,. 168,. 168) 

23 

.3 

.5 

.1 

.4 

(.015,. 060,. 075,. 015,. 060,. 075, 
.035,. 140,. 175,. 035,. 140,. 175) 

24 

.3 

.5 

.3 

.3 

(.045,. 045,. 060,. 045,. 045,. 060, 
.105,. 105,. 140,. 105,. 105,. 140) 

25 

.4 

.4 

.1 

.1 

(.016,. 016,. 128,. 024,. 024,. 192, 
.024,. 024,. 192,. 036,. 036,. 288) 

26 

.4 

.4 

.2 

.2 

( . 032 , . 032 , . 096 , . 048 , . 048 , . 144 , 
.048,. 048,. 144,. 072,. 072,. 216) 

27 

.4 

.5 

.1 

.2 

(.020, .040, . 140, .020, .040, . 140 
.030,. 060,. 210,. 030,. 060,. 210) 

28 

.4 

.5 

.2 

.3 

(.040,. 060,. 100,. 040,. 060,. 100, 
.060,. 090,. 150,. 060,. 090,. 150) 

29 

.5 

.5 

.1 

.3 

(.025,. 075,. 150,. 025,. 075,. 150, 
.025,. 075,. 150,. 025,. 075,. 150) 

30 

.5 

.5 

.2 

.4 

(.050,. 100,. 100,. 050,. 100,. 100, 
.050, . 100, . 100, .050, . 100, . 100) 

31 

.5 

.5 

1/3 

1/3 

(1/12,1/12,1/12,1/12,1/12,1/12, 

1/12,1/12,1/12,1/12,1/12,1/12) 

APPENDIX  C 


SAMPLE  SIZE  DESIGN 


TABLE 

k 

# CELLS 

N 

SAMPLE  SIZES 

fn 

AVE.  EXPECTED  VALUES 

TOTAL  #N' 

E2  *  2 

4 

4(4)36 

1(1)9 

9 

40(8)56 

10(2)14 

3 

M2  x  2 

4 

64(8)96 

16(2)24 

5 

2*3 

6 

12(6)30 

2(1)5 

4 

2*4 

8 

16(8)40 

2(1)5 

4 

48(16)96 

•  6(2)12 

4 

3*3 

9 

18(9)45 

2(1)5 

4 

• 

54(18)108 

6(2)12 

4 

2*5 

10 

20(10)100 

2(1)10 

9 

2*2*2 

8 

16(8)40 

2(1)5 

4 

48(16)96 

6(2)12 

4 

2*2*3 

12 

24(12)96 

2(1)8 

7 

|s2  *  2  *  2 

8 

24(16(104) 

3(2)13 

_ 

6 

E  -  Exact 
M  -  Monte  Carlo 


S  -  No  Second-Order  Interaction 


APPENDIX  D 


EXACT  2x2  PROGRAM 


C****  THIS  PROGRAM  CALCULATES  EXACT  LEVELS  OF  SIGNIFICANCE  (BASED 
C  ON  NOMINAL  LEVELS  .10,. 05,. 01)  FOR  THREE  ASYMPTOTIC 
C  CHI-SQUARE  STATISTICS  ( PEARSON, KULLBACK, GSR)  WHEN  USED  AS 
C  INDEPENDENCE  HYPOTHESIS  TEST  STATISTICS  IN  2X2  CONTINGENCY 
C****  TABLES  WITH  SAMPLE  SIZE  N  (DIVSIBLE  BY  FOUR). 

PROGRAM  EXCTA  (INPUT, OUTPUT, TAPE5- INPUT, TAPE6-OUTPUT) 

DIMENSION  A(3,3) 

COMMON  X(4) ,P(4) ,XN 
COMMON/ STAT/G2 , X2 , X2GSK1 
COMHON/HYP/FTOTAL, FSTAT( 3 , 3) , F 
WRITE(6, 15) 

15  FORMAT( 50X, ' EXACT  ALPHAS  FOR  CTA  STATISTICS',//) 

WRITE(6,25) 

25  FORMAT( 3 IX, 'NOMINAL  ALPHA  -  . 10' ,9X, 'NOMINAL  ALPHA  -  .05', 

+9X, 'NOMINAL  ALPHA  -  .01',/) 

WRITE(6,35) 

35  F0RMAT(5X, 'P  VECTOR' , 10X, 'N' ,5X, 'KULLBACK' , IX, 'PEARSON' , 

+3X, ' GSK1 ' , 2(5X, 'KULLBACK' , IX, 'PEARSON' , 3X, ' GSK1 ' ) , 4X, 'FTOTAL' ) 
C****  CYCLE  THROUGH  PROBABILITY  VECTORS  OF  DESIGN. 

DO  50  L-1,5 
P1D0T-FLQAT(L)*.1 
DO  40  M-L,5 
PDOTl-FLOAT(M)*.l 
P(1)-P1D0T*PD0T1 
P(2)"P1D0T*( 1.-PD0T1) 

P(3)-(1.-P1D0T)*PD0T1 
P(4)-(1.-P1D0T)*( 1.-PD0T1) 

C****  ENTER  SAMPLE  SIZES  IN  DO  LOOP  (INCREMENTED  BY  FOUR). 

DO  30  N-4,40,4 
XN-N 

C****  INTIALIZE  PARAMETERS:  FTOTAL  -  TOTAL  MULTINOMIAL  PROBABILITY, 
C****  FSTAT(I.J)  -  ACCUMULATED  MULTINOMIAL  REJECTION  PROBABILITIES. 
FTOTAL-O.O 
DO  20  1-1,3 
DO  10  J-1,3 
FSTAT(I, J)-0.0 

10  CONTINUE 

20  CONTINUE 

C****  ACCUMULATE  FSTAT  FOR  GROUP  El  (ALL  CELLS  EQUAL). 

DO  200  1-1,4 
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200  CONTINUE 
F-FMULK  ) 

CALL  STATS 
CALL  HYP ACC 

C****  ACCUMULATE  FSTAT  FOR  GROUP  E2  (BOTH  CORNERS  EQUAL). 

UO  300  I«0,(N/4-l) 

X(1)-FL0AT(I) 

X(4)-X(l) 

X(2)»(XN-(2.*X(l)))/2. 

X(3)-X(2) 

F-2.0*FMULT() 

CALL  STATS 
CALL  HYPACC 
300  CONTINUE 

IF(N.EQ.4)G0  TO  410 

C****  ACCUMULATE  FSTAT  FOR  GROUP  E3  (BOTH  SIDES  EQUAL). 

DO  400  I-l,(N/4-l) 

X(1)-FL0AT(I) 

X(2)-X(l) 

X(3)-(XN-(2.‘*X(l)))/2.0 

X(4)«X(3) 

F-FMULT( ) 

CALL  R0T4(F) 

CALL  STATS 
CALL  HYPACC 
400  CONTINUE 
410  DO  500  I-l,(N/2-l) 

X(l)-0.0 

X(4)-0.0 

X(2)-FL0AT(I) 

X(3)-XN-X(2) 

F-FMULT( ) 

CALL  RDT4(F) 

CALL  STATS 
CALL  HYPACC 
500  CONTINUE 

C****  ACCUMULATE  FSTAT  FOR  GROUP  E4  (ONE  SET  EQUAL  CORNERS). 
DO  550  I«l,(N/2-l) 

DO  540  J»0,((N-2*I-2)/2) 

X(1)-FL0AT(I) 

X(4)-X(l) 

X(2)-FL0AT(J) 

X( 3 )-XN-X( 1 )-X( 2 ) -X( 4 ) 

F-FMULTO 
CALL  R0T4(F) 

CALL  STATS 
CALL  HYPACC 
540  CONTINUE 
550  CONTINUE 

c****  ACCUMULATE  FSTAT  FOR  GROUP  0  (ALL  OTHERS). 

DO  610  >l,((N-2)/2) 

DO  600  J»(I+1),(N-I-1) 

X(1)«0.0 
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I 


X(2)-FL0AT(I) 

X(3)-FL0AT(J) 

X(4)-XN-X(2)-X(3) 

F-FMULT( ) 

CALL  R0T8(F) 

CALL  STATS 
CALL  HYPACC 
600  CONTINUE 
610  CONTINUE 

IF(N.EQ.4)GO  TO  700 
DO  660  I«l,((N-4)/4) 

DO  650  J-<I+l),((N-2*I-2)/2) 

X(l)-FLOAT(I) 

X(2)-X(l) 

X(3)-FL0AT(J) 

X(4)-XN-X(1)-X(2)-X(3) 

F-FMULT( ) 

CALL  ROT8(F) 

CALL  STATS 
CALL  HYPACC 
650  CONTINUE 
660  CONTINUE 

DO  690  I-l,((N-4)/4) 

DO  680  J-(I+l),((N-2*I-2)/2) 

DO  670  K-(J+1),(N-2*I-J-1) 

X(1)-FL0AT(I) 

X(2)-FL0AT(J) 

X(3)-FL0AT(K) 

X(4)-XN-X(1)-X(2)-X(3) 

F-FMULTO 
CALL  ROT8(F) 

CALL  STATS 
CALL  HYPACC 
670  CONTINUE 
680  CONTINUE 
690  CONTINUE 

C****  CALCULATE  EXACT  LEVEL  OF  SIGNIFICANCE  A(I,J)  FOR  EACH 

C****  STATISTIC(I)  AND  NOMINAL  LEVEL(J)  COMBINATION  AND  PRINT  RESULTS. 
700  DO  720  1-1,3 
DO  710  J-1,3 

A(I , J) -F  STAT( I , J ) / FTOTAL 
710  CONTINUE 
720  CONTINUE 

WRITE<6, 45)(P(I),I-1, 4), N,( (A(I, J), 1-1,3), J-1,3), FTOTAL 
45  FORMAT (2H  ( ,3(F3. 2, ' , ' ) ,F3. 2, ' ) ' , 3X, I3,5X,F6.4, 
+2(3X,F6.4),1X,3(3X,F6.4),1X,3(3X,F6.4),2X,F6.4) 

30  CONTINUE 
40  CONTINUE 
50  CONTINUE 
STOP 
END 

C****  FMULT  -  FUNCTION  TO  CALCULATE  MULTINOMIAL  PROBABILITIES. 

FUNCTION  FMULT () 


■j 
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v, 


s 

N 

X 

i-X 


V' 


COMMON  X(4),P(4),XN 

FMULI-1.0 

DO  100  1-1,4 

FMULT-FMULT* (P(I)**X(I))/ FACT( X( I ) ) 

100  CONTINUE 
FMULT-FMULT*  FACT ( XN) 

RETURN 

END 

C****  FACT  -  FUNCTION  TO  CALCULATE  FACTORIALS. 

FUNCTION  FACT(Z) 

FACT- 1.0 

IF(Z.EQ.O.O)GO  TO  125 
DO  120  1-2, IFIX(Z) 

FACT-FACT*FLOAT( I) 

120  CONTINUE 
125  RETURN 
END 

C****  HYP ACC  -  SUBROUTINE  TO  PERFORM  ALL  HYPOTHESIS  TESTS  AND 
C****  ACCUMULATE  MULTINOMIAL  REJECTION  PROBABILITIES. 

SUBROUTINE  HYPACC 
COMMON/ HYP/ FTOTAL , FSTAT( 3 , 3 ) , F 
COMMON/ STAT/ G2 , X2 , X2GSK1 
FTOTAL-FTOTAL+F 
IF(G2.GT.6. 63490)G0  TO  101 
IF(G2.GT. 3.84146)GO  TO  102 
IF(G2.GT.2.70554)GO  TO  103 
GO  TO  104 

101  FSTAT(1,3)-FSTAT(1,3HI 

102  FSTAT( 1 , 2)-FSTAT( 1 , 2)+F 

103  FSTAT( 1, 1)-FSTAT( 1, 1)+F 

104  IF(X2.GT.6.63490)GO  TO  106 
IF(X2.GT.3.84146)GO  TO  107 
IF(X2.GT.2.70554)GO  TO  108 
GO  TO  109 

106  FSTAT(2,3)-FSTAT(2,3)+F 

107  FSTAT(2,2)-FSTAT(2,2)+F 

108  FSTAT(2,1)-FSTAT(2,1)+F 

109  IF(X2GSK1.GT.6.63490)GO  TO  111 
IF(X2GSKl.GT.3.84146)GO  TO  112 
IF(X2GSK1.GT.2.70554)GO  TO  113 
GO  TO  114 

111  FSTAT(3,3)»FSTAT(3,3)+F 

112  FSTAT(3,2)-FSTAT(3,2)+F 

113  FSTAT(3,1)-FSTAT(3,1)+F 

114  RETURN 
END 

C****  STATS  -  SUBROUTINE  TO  CALCULATE  VALUES  FOR  THREE  CHI-SQUARE 
C****  STATISTICS:  PEARS0N(X2),  KULLBACK(G2) ,  AND  GSK(X2GSK1). 
SUBROUTINE  STATS 
DIMENSION  XM(4) 

COMMON  X(4) ,P(4) ,XN 
COMMON/ STAT/ G2,A2, X2GSK1 
X1D0T-X( 1 )+X( 2 ) 
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XDOT 1 -X( 1 )+X( 3 ) 

XM( 1)-X1D0T*XD0T1/XN 

XM( 2 ) -XI DOT* ( XN-XDOT 1 ) / XN 

XM(3)-(XN-X1D0T)*XD0T1/XN 

XM(  4 ) -( XN-X1DOT ) * ( XN-XDOT 1 ) / XN 

X2«((X(1)-XM(1))**2)*(1./XM(1)+1./XM(2)+1./XM(3)+1./XM(4)) 

G2-0.0 

DO  110  1-1,4 

IF(X(I).EQ.O.O)GO  TO  105 
G2-G2+2.0*X(I)*L0G(X(I)/XM(I)) 

GO  TO  110 
105  X(I)-0.25 
110  CONTINUE 

X2GSKl-( ( LOG( X( 1 ) ) -LOG( X( 2 ) ) -LOG( X( 3 ) )+LOG( X( 4 ) ) ) 
+**2)/(l./X(l)+l./X(2)+l./X(3)+l./X(4)) 

RETURN 

END 

C****  K0T8  -  SUBROUTINE  TO  ROTATE  CONTINGENCY  TABLE  THROUGH  EIGHT 
C****  SYMMETRIC  POSITIONS; 

SUBROUTINE  R0T8(F) 

COMMON  X(4),P(4),XN 
JJ-1 

130  DO  140  1-1,3 
X(l)-X(2) 

X(2)-X(4) 

X(4)-X(3) 

•  X(3)-XN-X( 1)-X(2)-X(4) 

F-F+FMULT( ) 

140  CONTINUE 

IF(JJ.EQ.O)GO  TO  150 
A-X(l) 

X(l)-X(2) 

X(2)-A 

X(3)-X(4) 

X(4)-XN-X(1)-X(2)-X(3) 

F-F+FMULT( ) 

JJ-0 

GO  TO  130 
150  RETURN 
END 

C****  R0T4  -  SUBROUTINE  TO  ROTATE  CONTINGENCY  TABLE  THROUGH  FOUk 
C****  SYMMETRIC  POSITIONS. 

SUBROUTINE  R0T4(F) 

COMMON  X(4) ,P(4) ,XN  * 

DO  170  1-1,3 
X(1)»X(2) 

X(2)-X(4) 

X(4)-X(3) 

v/3)-XN-X( 1)-X(2)-X(4) 

F-j+R'1ULT(  )  i 

170  CONTINUE 
RETURN 
END 


APPENDIX  E 


MONTE  CARLO  2x3  PROGRAM 


C****  THIS  PROGRAM  CALCULATES  ESTIMATES  OF  EXACT  LEVELS  OF 
C  SIGNIFICANCE  (BASED  ON  NOMINAL  LEVELS  .10,. 05,. 01)  FOR 
C  THREE  ASYMPTOTIC  CHI-SQUARE  STATISTICS  (PEARSON, KULLBACK, 

C  GSK)  WHEN  USED  AS  INDEPENDENT  HYPOTHESIS  TEST  STATISTICS 
C  IN  2X3  CONTINGENCY  TABLES.  THE  PROCEDURE  IS  A  MONTE  CARLO 

C  SIMULATION  USING  2000  RANDOMLY  GENERATED  TABLES  BASED  ON 

C  THE  MULTINOMIAL  DISTRIBUTION. 

C  THE  PROGRAM  IS  DIVIDED  INTO  FOUR  PARTS:  A  MAIN  PROGRAM  AND 

C  THREE  SUBROUTINES.  THE  MAIN  PROGRAM  CONTOLS  THE  CYCLING 

C  PROCESS  THROUGH  THE  VARIOUS  PROBABILITY  DESIGNS  AND  SAMPLE 

C  SIZES,  CALLS  THE  SUBROUTINES,  MAKES  THE  FINAL  CALCULATIONS 

C  FOR  THE  ESTIMATED  EXACT  LEVELS  OF  SIGNIFICANCE,  AND 
C****  PRINTS  THE  RESULTS.  THE  SUBROUTINES  ARE  DESCRIBED  BELOW. 

PROGRAM  MC2BY3( INPUT, OUTPUT, TAPE5- INPUT, TAPE6-OUTPUT) 

DIMENSION  A(3,3) 

DOUBLE  PRECISION  DSEED 
COMMON  X( 2000,6) , NT, N 
COMMON/ SG/ P ( 6 ) , DSEED 

COMMON/ STAT/G2(2000) ,X2(2000) , X2GSK1( 2000) 

COMMON/ SHYP/ HI ( 3 , 3 ) 

WRITE(6, 15) 

15  F0RMAT(53X, 'ESTIMATED  EXACT  ALPHAS  FOR  CTA  STATISTICS',//) 
WRITE(6,25) 

25  FORMAT(39X, 'NOMINAL  ALPHA  -  . 10' , 9X, 'NOMINAL  ALPHA  -  .05', 

+9X, 'NOMINAL  ALPHA  -  .01',/) 

WRITE(6,35) 

35  F0RMAT(9X,'P  VECTOR' , 14X, 'N' ,5X, 'KULLBACK' , IX, 'PEARSON' , 

+3X, ' GSK1 ' , 2( 5X , ' KULLBACK' , IX, ' PEARSON ' , 3X, ' GSK1 ' ) ) 

NT-2000 

XNT-NT 

C****  DSEED  -  RANDOM  NUMBER  GENERATOR  SEED  IN  RANGE 
C****  (l.DO  ,  2147483647. DO). 

DSEED- 1847 3. ODO 

c****  IND  -  INDICATOR  FOR  EQUIPROBABLE  TABLE. 

IND-0 

C****  CYCLE  THROUGH  PROBABILITY  VECTORS  OF  DESIGN. 

DO  60  1-1,5, 2 
PIDOT-FLOAT(I)*. 1 
P2D0T-1-P1D0T 
DO  50  J-l,(3-MOD(I,2)> 

PDOTl-FLOAT(J)*.! 


IF<M0D(I,2).EQ.1)II-J 

IF(MOD(I,2).EQ.O)II-(5*J-J**2)/2 

IF(MOD( I , 2) . EQ . 1 )  J  J- J+2 

IF(MOD( I, 2) .EQ.O) JJ-( J**2-5*J)/2+6 

DO  40  K-II,JJ,2 

PD0T2-FLQAT(K)*.l 

PDOT3-1-PDOT1-PDOT2 

P(1)-P1D0T*PD0T1 

P(2)-P1DOT*PDOT2 

P(3)-P1D0T*PD0T3 

P(4)-( 1 .-P1D0T)*PD0T1 

P(5)-(1.-P1D0T)*PD0T2 

P(6)-( 1.-P1D0T)*PD0T3 

C****  CYCLE  THROUGH  SAMPLE  SIZES,  CALL  SUBROUTINES,  CALCULATE 
C  ESTIMATES  OF  EXACT  LEVELS  OF  SIGNIFICANCE  A(L,M)  FOR  EACH 
C  STATISTIC(L)  AND  NOMINAL  LEVEL (M)  COMBINATION,  AND 
C****  PRINT  RESULTS. 

100  DO  30  N-12,30,6 
CALL  GEN 
CALL  STATS 
CALL  HYP 
DO  20  L-1,3 
DO  10  M-1,3 
A(L,M)-HI(L,M)/XNT 
10  CONTINUE 
20  CONTINUE 

WRITE(6, 45)(P(L),L-1, 6), N,((A(L,M), L-1,3), M-1,3) 

45  F0RMAT(2H  (,5(F3.2,# , '),F3.2, ')',3X,I3,5X,F6.4, 
+2(3X,F6.4), 1X,3(3X,F6.4) , 1X,3(3X,F6.4)) 

30  CONTINUE 

IF(IND.EQ.1)G0  TO  74 
40  CONTINUE 
50  CONTINUE 
60  CONTINUE 

C****  DESIGNATE  EQUIPROBABLE  TABLE. 

DO  70  L-1,6 
P(L)-1. 0/6.0 
70  CONTINUE 
IND-1 
GO  TO  100 
74  STOP 
END 

C****  GEN  -  SUBROUTINE  USED  TO  GENERATE  2000  RANDOM  MULTINOMIAL 
C  OBSERVATIONS.  USES  IMSL(1980),  SUBROUTINE  GGMTN  CODING 
C  MODIFIED  FOR  EFFICIENCY  AND  SUBROUTINE  GGBN.  ALL  TABLES 
C  ARE  CHECKED  FOR  USABILITY  BASED  ON  THE  "NO  ZERO  MARGINALS" 
C  CRITERION.  THE  PROCEDURE  IS  CONTINUED  UNTIL  2000  USABLE 
C**n*  TABLES  HAVE  BEEN  GENERATED. 

SUBROUTINE  GEN 
DOUBLE  PRECISION  DSEED 
DIMENSION  IB( 1) 

COMMON  X(2000,6),NT,N 
COMMON/ SG/ P ( 6 ) , DSEED 


DO  90  L-l.NT 
75  NUSED-0 
PLEFT-1.0 
DO  80  MM-1,5 
M-7-MM 

PR-P(M)/PLEFT 

PLEFT-PLEFT-P(M) 

NLEFT-N-NUSED 

CALL  GGBN( DSEED , 1 , NLEFT , PR , IB ) 

X(L,M)«IB(1) 

NUSED-NUSED+IB(1) 

80  CONTINUE 

X(L,1)-N-NUSED 

IF(X(L,1)+X(L,4).EQ.0.0)G0  TO  75 
IF(X(L,2)+X(L,5).EQ.0.0)G0  TO  75 
IF(X(L,3)+X(L,6).EQ.0.0)G0  TO  75 
IF(X(L,1)+X(L,2)+X(L,3).EQ.O.O)GO  TO  75 
IF(X(L,4)+X(L,5)+X(L,6).EQ.O.O)GO  TO  75 
90  CONTINUE 
RETURN 
END 

C****  STATS  -  SUBROUTINE  TO  CALCULATE  THE  VALUES  OF  THREE  CHI-SQUARE 
C****  STATISTICS  PEARSON ( X2 ) ,  KULLBACK(G2) ,  AND  GSK(X2GSK1). 
SUBROUTINE  STATS 
DIMENSION  XM(6) ,XINV(6) 

COMMON  X(2000,6),NT,N 

COMMON/ STAT/ G2 ( 2000 ) , X2 ( 2000 ) , X2GSK1 ( 2000 ) 

XN-N 

DO  120  L-l.NT 
X1D0T-X(L, 1)+X(L,2)+X(L,3) 

X2DOT-X(L,4)+X(L,5)+X(L,6) 

XD0T1-X( L, 1 )+X( L , 4 ) 

XDOT2-X(L,2)+X(L,5) 

XDOT3-X(L,3)+X(L,6) 

XM( 1)-X1D0T*XD0T1/XN 

XM(2)-X1D0T*XD0T2/XN 

XM(3)-X1D0T*XD0T3/XN 

XM(4)-X2D0T*XD0T1/XN 

XM(5)-X2DOT*XDQT2/XN 

XM( 6)-X2DOT*XDOT3/XN 

X2(L)-0.0 

G2(L)-0.0 

DO  110  M-1,6 

X2(L)-X2(L)+((X(L,M)-XM(M))**2)/XM(M) 

IF  (X(L,M).EQ.O.O)GO  TO  105 
G2(L)-G2(L)+2.0*X(L,M)*L0G(X(L,M)/XM(M)) 

GO  TO  107 

105  X(L,M)-1. 0/6.0 
107  XINV(M)-1./X(L,M) 

110  CONTINUE 

Fll-LOG(X(L,l)*X(L,6)/(X(L,3)*X(L,4))) 

F12-LOG(X(L,2)*X(L,6)/(X(L.3)*X(L,5))) 

SUM3-XINV( 3 )+XINV( 6 ) 
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SUH1-XINV(  1  )+XINV(  4)+SUM3 
SUM2-XINV(  2)+XINV(  5)+SUM3 

X2GSK1(L)-((F11**2)*SUM2+(F12**2)*SUM1-2.*F11*F12*SUM3)/ 
+(  SUM1*SUM2-SUM3**2 ) 

120  CONTINUE 
RETURN 
END 

C****  HYP  -  SUBROUTINE  TO  PERFORM  ALL  HYPOTHESIS  TESTS  AND 
C  ACCUMULATE  THE  NUMBER  OF  REJECTIONS  HI(L,M)  FOR  EACH 
C****  STATISTIC(L)  AND  NOMINAL  LEVEL(M)  COMBINATION. 

SUBROUTINE  HYP 
COMMON  X(2000,6),NT,N 

COMMON/ STAT/ G2( 2000 ) , X2( 2000 ) , X2GSK1 ( 2000 ) 

COMMON/ SHYP/ HI ( 3 , 3 ) 

DO  150  L-1,3 
DO  140  M-1,3 


140 

150 


151 

152 

153 

154 


156 

157 

158 

159 


161 

162 

163 

164 
440 


HI(L,M)-O.U 
CONTINUE 
CONTINUE 
DO  440  L-l.NT 

IF(G2(L).GT. 9. 21034)00  TO  151 
IF(G2(L).GT.5.99147)GO  TO  152 
IF(G2(L).GT.4.60517)GO  TO  153 
GO  TO  154 

HI(1,3)-HI(1,3)+1.0 
HI(1,2)-HI(1,2)+1.0 
HI(1,1)-HI(1,1)+1.0 
IF(X2(L) .GT.9.21034)G0  10  156 
IF(X2(L).GT.5.99147)GO  TO  157 
IF(X2(L).GT.4.60517)G0  TO  158 
GO  TO  159 

HI(2,3)-HI(2,3)+1.0 
HI(2,2)-HI(2,2)+1.0 
HI(2, 1)*HI(2, 1)+1.0 
IF(X2GSK1<L).GT.9.21034)GO  TO  161 
IF(X2GSKl(L).GT.5.99147)GO  TO  162 
IF(X2GSK1(L).GT.4.60517)GO  TO  163 
GO  TO  164 

HI(3,3)-HI(3,3)+1.0 

HI(3,2)-HI(3,2)+1.0 

HI(3tl)-HI(3,l)-H.O 

CONTINUE 

CONTINUE 

RETURN 

END 
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MINIMUM  AND  MAXIMUM  SIGNIFICANCE  LEVELS 
FOR  MINIMUM  CELL  EXPECTATION  INTERVALS(MCEI) 
TABLE:  2X2 


NOMINAL  ALPHA  -  .10 

MCE  I  KULLBACK  PEARSON  GSK 

[0.5,1.0)  (.0595, .1505)  ( .0674, . 1431)  ( .0000, .0660) 

11.0,1.5)  (.1077, .1694)  ( .0650, . 1429)  ( .0000, .0560) 

(1.5, 2.0)  (.1308, .1734)  (.0690, . 1426)  ( .0071, .0601) 

[2. 0,2. 5)  (.1266, .1732)  ( .0952, . 1472)  ( .0074, .0672) 

[2. 5, 3.0)  (.1204, .1409)  ( .0983, . 1160)  ( .0336, .0745) 

(3.0.3.5)  (.1090, .1716)  ( .0905, . 1517)  ( .0490, .0787) 

(3. 5, 4.0)  (.0915, .1265)  ( .0835, . 1145)  ( .0530, .0809) 

[4. 0,4. 5)  (.1110,. 1281)  (.0995,. 1193)  ( .0605, .0871) 

(4. 5, 5.0)  (.1040, .1168)  (. 1010, . 1104)  ( .0735, .0900) 


NOMINAL  ALPHA  -  .05 

MCEI  KULLBACK  PEARSON  GSK 

[0.5, 1.0)  (.0271, .1431)  (.0175, .1431)  ( .0000, .0395) 

[1.0, 1.5)  (.0439, .1429)  ( .0258, . 1429)  ( .0000, .0245) 

11.5,2.0)  (.0600,. 0929)  (.0310, .0606)  ( .0000, .0330) 

[2.0, 2. 5)  (.0690, .0981)  ( .0386, .0634)  ( .0000, .0267) 

[2. 5,3.0)  (.0645,. 0805)  ( .0405, .0635)  ( .0064, .0305) 

[3. 0,3. 5)  (.0575, .0838)  (.0410,-0670)  ( .0120, .0291) 

[3. 5, 4.0)  (.0425,-0680)  ( .0375, .0565)  ( .0110, .0355) 

[4. 0,4. 5)  (.0585, .0789)  ( .0494, .0706)  ( .0157, .0361) 

[4.5.5.0)  (.0565, .0660)  ( .0490, .0580)  ( .0260, .0372) 


NOMINAL  ALPHA  -  .01 

MCEI  KULLBACK  PEARSON  GSK 

[0.5,1.0)  (.0000, .0143)  (.0000, .0225)  ( .0000, .0090) 

[1.0, 1.5)  (.0000, .0171)  (.0000, .0132)  ( .0000, .0035) 

[1.5, 2.0)  ( .0060, .0193)  ( .0028, .0140)  ( .0000, .0070) 

[2.0.2.5)  (.0074, .0200)  ( .0036, .0105)  ( .0000, .0046) 

12.5,3.0)  (.0110, .0185)  ( .0052, .0135)  ( .0000, .0085) 

[3.0,3.5)  (.0146, .0240)  ( .0063, .0113)  ( .0000, .0060) 

[3.5.4.0)  (.0105, .0180)  ( .0065, .0105)  (.0000,-0040) 

[4.0,4. 5)  (.0136, .0195)  ( .0045, .0118)  ( .0000, .0044) 

[4.5.5.0)  (.0120, .0170)  ( .0088, .0105)  (.0005,-0040) 

[4.5,5.0)  (.0120, .0170)  ( .0088, .0105)  (.0005,-0040) 


MINIMUM  AND  MAXIMUM  SIGNIFICANCE  LEVELS 
FOR  MINIMUM  CELL  EXPECTATION  INTERVALS (MCE I) 
TABLE:  2X3 


NOMINAL  ALPHA  -  .10 

MCE I  KULLBACK  PEARSON  GSK 

10.5,1.0)  (.0675, .1550)  ( .0550, . 1080)  ( .0000, .0525) 

[1.0, 1.5)  (.1185, .1715)  (.0730, .1115)  ( .0000, .0600) 

[1.5, 2.0)  (.1140, .1720)  (.0830, .1125)  ( .0020, .0725) 

[2. 0,2. 5)  (.1195, .1685)  ( .0905, . 1085)  ( .0005, .0670) 

[2.5, 3.0)  (.1090, .1430)  (.0875,-1075)  ( .0360, .0765) 

[3.0.3.5)  (.1050, .1515)  ( .0935, . 1090)  ( .0190, .0685) 

[3. 5, 4.0)  (.1045, .1420)  (.0925,-1180)  ( .0360, .0875) 

[4. 0,4. 5)  (.1115, .1305)  ( .0970, . 1110)  ( .0370, .0830) 

[4.5, 5.0)  (.0960, .1315)  (.0895, .1165)  ( .0605, .0765) 


NOMINAL  ALPHA  -  .05 

MCEI  KULLBACK  PEARSON  GSK 

[0.5, 1.0)  (.0290,. 0745)  ( .0155, .0605)  ( .0000, .0285) 

[1.0, 1.5)  (.0545,-0890)  ( .0305, .0540)  ( .0000, .0245) 

11.5,2.0)  (.0540, .0905)  ( .0280, .0520)  ( .0005, .0265) 

[2. 0,2.5)  (.0650,. 0930)  ( .0405, .0550)  ( .0000, .0270) 

[2.5,3.0)  (.0550, .0770)  ( .0400, .0560)  ( .0115, .0325) 

13.0,3.5)  (.0560,-0870)  ( .0430, .0535)  ( .0015, .0240) 

[3.5, 4.0)  (.0550, .0785)  ( .0390, .0580)  ( .0100, .0395) 

[4. 0,4. 5)  (.0625,. 0680)  ( .0430, .0560)  ( .0075, .0430) 

[4. 5, 5.0)  (.0460, .0695)  ( .0385, .0545)  ( .0155, .0295) 


NOMINAL  ALPHA  -  .01 

MCEI  KULLBACK  PEARSON  GSK 

10.5,1.0)  (.0025, .0165)  ( .0015, .0230)  ( .0000, .0050) 

[1.0, 1.5)  (.0070, .0215)  (.0020, .0140)  ( .0000, .0020) 

[1.5, 2.0)  (.0055, .0175)  ( .0020, .0115)  ( .0000, .0040) 

[2.0,2.5)  (.0095, .0255)  ( .0025, .0120)  ( .0000, .0030) 

12.5,3.0)  (.0090, .0195)  ( .0060,  .0115)  ( .0000, .0040) 

[3.0,3.5)  (.0090, .0225)  ( .0050, .0110)  ( .0000, .0045) 

[3. 5, 4.0)  (.0100, .0230)  ( .0055, .0155)  ( .0000, .0090) 

(4.0.4.5)  (.0140, .0185)  ( .0070, .0125)  ( .0000, .0055) 

14.5,5.0)  (.0115, .0190)  (.0065,-0105)  ( .0010, .0045) 

[4.5,5.0)  (.0115, .0190)  ( .0065, .0105)  ( .0010, .0045) 


•* 
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MINIMUM  AND  MAXIMUM  SIGNIFICANCE  LEVELS 
FOR  MINIMUM  CELL  EXPECTATION  INTERVALS (MCE I ) 
TABLE:  2X4 


MCEI 

[0. 5,1.0) 
[1.0t1.5) 
[1.5, 2.0) 
12.0,2.5) 
[2. 5, 3.0) 
[3. 0,3. 5) 
13.5,4.0) 
(4.0.4.5) 
[4. 5, 5.0) 


NOMINAL  ALPHA  -  .10 


KULLBACK 
(.0775, .1610) 
(.1070,. 1690) 
(.1185, .1710) 
(.1265, .1765) 
(.1185, .1470) 
(.1240,. 1680) 
(.1145,. 1330) 
(.1245,. 1490) 
(.1010,. 1260) 


PEARSON 
(.0590,. 1070) 
(.0575,. 1060) 
(.0725,. 1055) 
(.0770,. 1005) 
(.0925,. 1075) 
(.0895, .1110) 
(.0925, .1115) 
(.0945,. 1085) 
(.0910, .1055) 


GSK 

(.0000,. 0440) 
(.0005,. 0390) 
(.0000,. 0505) 
(.0000,. 0530) 
(.0275,. 0595) 
(.0065,. 0635) 
(.0465,. 0715) 
(.0285,. 0705) 
(.0420,. 0635) 


. -MINAL  ALPHA  -  .05 


MCEI 

[0.5,1. 0)  ( 
[1.0,1. 5)  ( 
[1. 5,2.0)  ( 
12.0,2.5)  ( 
12.5,3.0)  ( 
13.0,3.5)  ( 
[3.5,4. 0)  ( 
[4. 0,4. 5)  ( 
14.5,5.0)  ( 


KULLBACK 
.0250,. 0810) 
.0510,. 0900) 
.0550,. 0940) 
.0660,. 1025) 
.0620,. 0765) 
.0645,. 0900) 
.0555,. 0760) 
.0655,. 0830) 
.0565, .0705) 


PEARSON 
(.0150,. 0640) 
(.0225,. 0505) 
(.0285,. 0565) 
(.0365,. 0520^ 
(.0375,. 0510) 
(.0375,. 0540) 
(.0415,. 0605) 
(.0445,. 0540) 
(.0455,. 0560) 


GSK 

(.0000,. 0235) 
(.0000,. 0105) 
(.0000,. 0210) 
(.0000,. 0190) 
(.0045,. 0235) 
(.0010,. 0235) 
(.0140,. 0275) 
(.0060,. 0285) 
(.0100,. 0265) 


NOMINAL  ALPHA  -  .01 


MCEI 

10.5,1.0)  ( 
11.0,1.5)  ( 
[1.5, 2.0)  ( 
12.0,2.5)  ( 
12.5,3.0)  ( 
13.0,3.5)  ( 
13.5,4.0)  ( 
14.0,4.5)  ( 
[4.5, 5.0)  ( 
14.5,5.0)  ( 


KULLBACK 
.0030,. 0195) 
.0080,. 0205) 
.0070, .0205) 
.0120,. 0230) 
.0115, .0195) 
.0125,. 0265) 
.0095, .0195) 
.0155, .0255) 
.0075, .0165) 
.0075,. 0165) 


PEARSON 
(.0015, .0235) 
(.0015, .0100) 
(.0005,. 0135) 
(.0015, .0120) 
(.0040, .0115) 
(.0050, .0170) 
(.0050, .0115) 
(.0055, .0105) 
(.0055, .0115) 
(.0055, .0115) 


GSK 

(.0000,. 0015) 
(.0000,. 0010) 
(.0000,. 0025) 
(.0000,. 0015) 
(.0000,. 0035) 
(.0000,. 0040) 
(.0000,. 0040) 
(.0000,. 0045) 
(.0005,. 0035) 
(.0005,-0035) 


m 


254 


MCE  I 
[0.5,1 
[1.0,1 
[1-5,2 
[2.0,2 
[2.5,3 
[3.0,3 
[3.5,4 
[4.0,4 
[4.5,5 


MCE1 

[0.5,1 

[1.0,1 

[1.5,2 

12.0,2 

[2.5,3 

[3.0,3 

[3.5,4 

[4.0,4 

[4.5,5 


MCE1 

[0.5,1 

[1.0,1 

[1.5,2 

[2.0,2 

[2.5,3 

[3.0,3 

[3.5,4 

[4.0,4 

[4.5,5 

[4.5,5 


MINIMUM  AND  MAXIMUM  SIGNIFICANCE  LEVELS 
FOR  MINIMUM  CELL  EXPECTATION  INTERVALS(MCEI) 
TABLE:  3X3 


NOMINAL  ALPHA  -  .10 


KULLBACK 

PEARSON 

GSK 

0) 

(.0600, .1695) 

(.0695, .1150) 

(.0000,. 0000) 

5) 

(.1100, .1765) 

(.0840,. 1070) 

(.0000,. 0000) 

0) 

(.1295,. 1925) 

(.0880,. 1130) 

(.0000,. 0000) 

5) 

(.1075,. 1500) 

(.0860,. 1150) 

(.0000,. 0000) 

0) 

(.1125, .1720) 

(.0925, .1120) 

(.0000,. 0000) 

5) 

(.1155, .1365) 

(.0875, .1020) 

(.0000,. 0000) 

0) 

(.1030,. 1435) 

(.0905,. 1120) 

(.0000,. 0000) 

5) 

(.1015, .1350) 

(.0890,. 1060) 

(.0000,. 0000) 

0) 

(.1170, .1260) 

(.0940,. 0990) 

(.0000,. 0000) 

NOMINAL  ALPHA  -  . 

05 

KULLBACK 

PEARSON 

GSK 

0) 

(.0245,. 0920) 

(.0335,. 0595) 

(.0000,. 0000) 

5) 

(.0445,. 0965) 

(.0395,. 0580) 

(.0000,. 0000) 

0) 

(.0660,. 1035) 

(.0390,. 0545) 

(.0000,. 0000) 

5) 

(.0600,. 0920) 

(.0375, .0545) 

(.0000,. 0000) 

0) 

(.0595,. 1005) 

(.0430,. 0550) 

(.0000,. 0000) 

5) 

(.0610,. 0700) 

(.0430,. 0455) 

(.0000,. 0000) 

0) 

(.0530,. 0795) 

(.0410,. 0530) 

(.0000,. 0000) 

5) 

(.0490,. 0700) 

(.0400,. 0540) 

(.0000,. 0000) 

0) 

(.0605,. 0645) 

(.0425,. 0480) 

(.0000,. 0000) 

NOMINAL  ALPHA  -  . 

01 

KULLBACK 

PEARSON 

GSK 

0) 

(.0035,. 0160) 

(.0040,. 0210) 

(.0000,. 0000) 

5) 

(.0035, .0185) 

(.0050, .0175) 

(.0000,. 0000) 

0) 

(.0100,. 0205) 

(.0035, .0115) 

(.0000,. 0000) 

5) 

(.0100, .0195) 

(.0055, .0150) 

(.0000,. 0000) 

0) 

(.0090,. 0195) 

(.0060,. 0100) 

(.0000,. 0000) 

5) 

(.0120, .0150) 

(.0050,. 0080) 

(.0000,. 0000) 

0) 

(.0100, .0210) 

(.0075, .0110) 

(.0000,. 0000) 

5) 

(.0110, .0180) 

(.0065, .0125) 

(.0000,. 0000) 

0) 

(.0155, .0185) 

(.0080,. 0085) 

(.0000,. 0000) 

0) 

(.0155, .0185) 

(.0080,. 0085) 

(.0000,. 0000) 

MINIMUM  AND  MAXIMUM  SIGNIFICANCE  LEVELS 
FOR  MINIMUM  CELL  EXPECTATION  INTERVALS(MCEI) 
TABLE:  2X5 


MCE  I 

10.5,1.0) 
[1.0, 1.5) 
11.5,2.0) 
[2.0, 2. 5) 
12.5,3.0) 
[3.0,3.5) 
[3. 5, 4.0) 
14.0,4.5) 
[4. 5, 5.0) 


NOMINAL  ALPHA  -  .10 

KULLBACK  PEARSON  GSK 

(.0750,. 1865)  (.0745,. 1125)  ( .0000, .0230) 

(.1135,. 1890)  (.0660,. 1095)  ( .0000, .0330) 

(.1285,. 1855)  (.0740,. 1105)  ( .0005, .0400) 

(.1210,. 1845)  (.0845,. 1080)  ( .0000, .0415) 

(.1245, .1740)  (.0855, .1025)  ( .0105, .0575) 

(.1135, .1590)  (.0875, .1035)  ( .0095, .0590) 

(.1155, .1490)  (.0965, .1055)  ( .0175, .0650) 

(.1110, .1510)  (.0835, .1080)  ( .0265, .0675) 

(.1200,. 1270)  (.0995,. 1040)  ( .0405, .0625) 


MCEI 

10.5,1.0) 
11.0,1.5) 
[1. 5,2.0) 
[2.0,2.5) 
12.5,3.0) 
[3. 0,3. 5) 
13.5,4.0) 
[4. 0,4. 5) 
[4. 5, 5.0) 


MINAL  ALPHA  -  .05 


KULLBACK 
(.0320,. 0950) 
(.0485,-0970) 
(.0570,. 0945) 
(.06 70,. 1020) 
(.0660,. 0850) 
( .0660, £0955) 
(.0635,. 0840) 
(.0580,. 0880) 
(.0640,. 0705) 


PEARSON 
(.0305,. 0770) 
(.0165,. 0650) 
(.0255,. 0525) 
(.0330,. 0530) 
(.0345, .0545) 
(.0330,. 0550) 
(.0410,. 0560) 
(.0395,. 0580) 
(.0455,. 0500) 


GSK 

(.0000,. 0075) 
(.0000,-0155) 
(.0000,. 0115) 
(.0000,. 0150) 
(.0020,. 0235) 
(.0005,. 0255) 
(.0040,. 0270) 
(.0045,. 0295) 
(.0100,. 0195) 


NOMINAL  ALPHA  -  .01 


MCEI  KULLBACK 

10.5,1.0)  (.0025, .0180) 

[1.0, 1.5)  (.0085, .0230) 

11.5,2.0)  (.0105, .0230) 

[2.0, 2. 5)  (.0125, .0225) 

[2. 5, 3.0)  (.0130, .0195) 

[3.0, 3. 5)  (.0125, .0220) 

13.5,4.0)  (.0135, .0190) 

[4. 0,4. 5)  (.0105, .0265) 

[4.5, 5.0)  (.0130,. 0150) 

[4. 5, 5.0)  (.0130, .0150) 


PEARSON  GSK 

(.0025,. 0230)  (.0000, .0005) 

(.0000,. 0220)  (.0000,. 0010) 

(.0020,. 0120)  (.0000,. 0015) 

(.0020,. 0120)  (.0000,. 0020) 

(.0025,. 0090)  (.0000,. 0020) 

(.0050,. 0110)  (.0000,. 0030) 

(.0065,. 0100)  (.0005,. 0020) 

(.0055,. 0105)  (.0000,. 0030) 

(.0070,. 0090)  (.0000,. 0015) 

(.0070,. 0090)  (.0000,. 0015) 


-C 

. ,, -v-\  .\  j-. 

V* 

Y 

ft 

v 

256 

r 

* 

« 

MINIMUM  AND  MAXIMUM  SIGNIFICANCE 

LEVELS 

FOR  MINIMUM  CELL  EXPECTATION  INTERVALS (MCE I)  1 

TABLE: 

S2X2X2 

1 

NOMINAL  ALPHA  -  . 

10 

MCE  I  • 

KULLBACK 

PEARSON 

GSK 

C*, 

[0.5, 1.0) 

(.1390, .2675) 

(.1000,-1520) 

(.0080,. 0310) 

.  - 

11.0,1.5) 

(.1325, .2345) 

(.1000,. 1455) 

(.0085,. 0455) 

i 

[1.5, 2.0) 

(.1285, .1790) 

(.1020,. 1430) 

(.0180,. 0650) 

3 

[2. 0,2. 5) 

(.1260, .1730) 

(.1045,. 1305) 

(.0285,. 0730) 

[2. 5, 3.0) 

(.1270, .1615) 

(.1075,. 1225) 

(.0385,. 0780) 

*  ” 

[3.0, 3. 5) 

(.1120,. 1480) 

(.1005,. 1230) 

(.0580,. 0885) 

13.5,4.0) 

(.1225,. 1285) 

(.1135, .1170) 

(.0670,. 0940) 

[4. 0,4. 5) 

(.1200,. 1295) 

(.1080,. 1245) 

(.0775,. 1080) 

■* 

*-* 

[4.5,5.0) 

(.1100,. 1205) 

(.1080, .1155) 

(.0955,. 0980) 

r-\ 

A 

NOMINAL  ALPHA  -  . 

05 

’5 

MCE  I 

KULLBACK 

PEARSON 

GSK 

\  \i: 

[0.5, 1.0) 

(.0600,. 1130) 

(.0475,. 0755) 

(.0005,-0060) 

I 

11.0,1.5) 

(.0605,. 1060) 

(.0455,-0695) 

(.0010,. 0160) 

w 

rj 

11.5,2.0) 

(.0660,. 0980) 

(.0460,. 0780) 

(.0030,-0250) 

[2. 0,2. 5) 

(.0610,. 0910) 

(.0510,. 0660) 

(.0035,. 0290) 

fi 

[2. 5, 3.0) 

(.0705,. 0870) 

(.0510, .0610) 

(.0090,-0340) 

*,  * 

[3.0, 3. 5) 

(.0630,. 0885) 

(.0525,-0665) 

(.0185,. 0360) 

[3. 5, 4.0) 

(.0625, .0715) 

(.0575, .0585) 

(.0255,. 0455) 

i 

[4. 0,4. 5) 

(.0640, .0715) 

(.0505,. 0660) 

(.0290,. 0495) 

p» 

P 

[4. 5, 5.0) 

(.0515,. 0620) 

(.0485,. 0565) 

(.0380,. 0435) 

!;n 

J-A 

NOMINAL  ALPHA  -  . 

01 

:;<  .v 

MCE  I 

KULLBACK 

PEARSON 

GSK 

% 

[0.5,1.0) 

(.0095,. 0255) 

(.0060,. 0180) 

(.0000,. 0000) 

[1.0, 1.5) 

(.0105,-0255) 

(.0050,. 0170) 

(.0000,. 0015) 

11.5,2.0) 

(.0105,. 0220) 

(.0060, .0150) 

(.0000,. 0045) 

-M 

V 

[2. 0,2. 5) 

(.0105, .0240) 

(.0070,. 0160) 

(.0000,. 0050) 

12.5,3.0) 

(.0135, .0225) 

(.0085,. 0160) 

(.0000,-0040) 

[3. 0,3. 5) 

(.0145,. 0230) 

(.0090,. 0165) 

(.0000,. 0040) 

[3. 5, 4.0) 

(.0155, .0200) 

(.0090, .0145) 

(.0020,. 0070) 

V 

[4.0,4.5) 

(.0150, .0205) 

(.0120, .0160) 

(.0025,. 0100) 

%  • 

14.5,5.0) 

(.0105, .0180) 

(.0085,. 0135) 

(.0045,. 0080) 

%»  -  i^‘ 

* 

T-* 

14.5,5.0) 

(.0105,. 0180) 

(.0085,. 0135) 

(.0045,. 0080) 

*  J 

y 

^  -V 

s 

V* 

MINIMUM  AND  MAXIMUM  SIGNIFICANCE  LEVELS 
FOR  MINIMUM  CELL  EXPECTATION  INTERVALS (MCE I) 
TABLE:  2X2X2 


MCE  I 

[0.5.1.0) 
[1.0, 1.5) 
11.5,2.0) 
[2. 0,2. 5) 
12.5,3.0) 
13.0,3.5) 
[3. 5, 4.0) 
[4. 0,4. 5) 
[4. 5, 5.0) 


NOMINAL 

KULLBACK 

(.0950,. 1620)  (, 

(.1090, .1655)  (. 

(.1180, .1735)  (. 

(.1100, .1910)  (. 

(.1145, .1265)  (. 

(.1100, .1620)  (. 
(.1210, .1425)  (. 

(.1140,. 1450)  (. 

(.1120, .1220)  (. 


ALPHA  -  . 

PEARSON 
0760,. 1040) 
0835,. 1095) 
0870,. 1020) 
0845,. 1120) 
0850,. 0955) 
0865,. 1180) 
0950,. 1075) 
0950,. 1005) 
0920,. 1090) 


GSK 

(.0005,. 0360) 
(.0010,. 0435) 
(.0025,. 0475) 
(.0030,-0565) 
(.0325,. 0530) 
(.0140,-0670) 
(.0470,. 0635) 
(.0255,. 0670) 
(.0520,-0775) 


MCE  I 

[0.5, 1.0) 
[1.0, 1.5) 
11.5,2.0) 
[2.0, 2.5) 
[2.5, 3.0) 
[3. 0,3. 5) 
[3. 5, 4.0) 
[4. 0,4. 5) 
14.5,5.0) 


NOMINAL 

KULLBACK 

(.0395, -0835)  (. 

(.0530,. 0860)  (. 

(.0605,-0980)  (, 

(.0565,. 0965)  (. 

(.0580, .0710)  (. 

(.0585, .0905)  (. 

(.0625, .0775)  (. 

(.0550,. 0835)  (. 

(.0590,. 0640)  (. 


ALPHA  -  .05 
PEARSON 

0330,. 0585)  ( 
0375,. 0545)  ( 
0375,. 0580)  ( 
0425,. 0585)  ( 
0365,. 0535)  ( 
0415, .0620)  ( 
0490,. 0520)  ( 
0400,-0535)  ( 
0425,. 0520)  ( 


GSK 

.0000,-0150) 
.0000,-0180) 
.0005, .C215) 
.0005,. 0220) 
.0095,. 0235) 
.0050, .0310) 
.0145,. 0270) 
.0075,. 0300) 
.0160,. 0375) 


MCE  I 

10.5,1.0) 
11.0,1.5) 
[1.5, 2.0) 
12.0,2.5) 
[2. 5, 3.0) 
[3. 0,3. 5) 
13.5,4.0) 
14.0,4.5) 
[4.5,5.0) 
[4.5.5.0) 


NOMINAL 

KULLBACK 

(.0050, .0175)  (. 

(.0085,. 0180)  (. 

(.0075, .0195)  (. 

(.0100, .0240)  (. 

(.0105, .0155)  (, 

(.0135, .0250)  (. 

(.0120, .0160)  (. 
(.0105, .0195)  (. 

(.0120, .0140)  (. 

(.0120, .0140)  (. 


ALPHA  -  .01 
PEARSON 

0065,-0195)  ( 
0060, .0135)  ( 
0055, .0125)  ( 
0060,. 0135)  ( 
0070,. 0100)  ( 
0080,. 0120)  ( 
0075,. 0100)  ( 
0075, .0125)  ( 
0060,. 0095)  ( 
0060,. 0095)  ( 


GSK 

.0000,. 0015) 
.0000,. 0025) 
.0000,. 0045) 
.0000,. 0030) 
.0005,. 0030) 
.0000,. 0020) 
.0010,. 0035) 
.0000,. 0070) 
.0010,. 0040) 
.0010,. 0040) 


MINIMUM  AND  MAXIMUM  SIGNIFICANCE  LEVELS 
FOR  MINIMUM  CELL  EXPECTATION  INTERVALS(MCEI) 
TABLE:  2X2X3 


NOMINAL  ALPHA  -  .10 

MCEI  KULLBACK  PEARSON  GSK 

[0.5, 1.0)  (.0475, .1150)  ( .0885, . 1350)  (. 0000 ,. 0000 ) 

[1.0, 1.5)  (.0670,-1190)  (.0905, .1345)  ( .0000, .0000) 

[1.5, 2.0)  (.0580, .1350)  ( .0885, . 1265)  ( .0000, .0000) 

[2. 0,2. 5)  (.0775, .1385)  ( .0930, . 1175)  ( .0000, .0000) 

[2. 5, 3.0)  (.0860, .1475)  (.0985, .1090)  ( .0000, .0000) 

[3. 0,3. 5)  (.0850, .1325)  ( .0890, . 1060)  ( .0000, .0000) 

[3. 5, 4.0)  (.0805, .0970)  ( .0785, .0970)  ( .0000, .0000) 

[4. 0,4. 5)  (.1025,. 1260)  ( .0855, . 1215)  ( .0000, .0000) 

[4. 5, 5.0)  (.1320,-1405)  ( .0985, . 1010)  ( .0000, .0000) 


NOMINAL  ALPHA  -  .05 

MCEI  KULLBACK  PEARSON  GSK 

10.5,1.0)  (.0195,. 0530)  ( .0470, .0990)  ( .0000, .0000) 

[1.0,1. 5)  (.0265, .0560)  ( .0460, .0835)  (.0000, .0000) 

[1.5, 2.0)  (.0195, .0735)  ( .0455, .0860)  ( .0000, .0000) 

[2.0.2.5)  (.0360, .0780)  ( .0460, .0695)  (.0000,-0000) 

[2. 5, 3.0)  (.0390,. 0770)  ( .0550, .0670)  (. 0000 ,. 0000 ) 

[3.0, 3.5)  (.0410, .0665)  ( .0445, .0620)  ( .0000, .0000) 

13.5,4.0)  (.0355,. 0445)  ( .0355, .0640)  ( .0000, .0000) 

[4. 0,4. 5)  (.0440,. 0600)  ( .0400, .0755)  ( .0000, .0000) 

[4. 5, 5.0)  (.0625,-0680)  ( .0455, .0485)  ( .0000, .0000) 


NOMINAL  ALPHA  -  .01 

MCEI  KULLBACK  '  PEARSON  GSK 

[0.5, 1.0)  (.0020,. 0080)  ( .0130, .0520)  ( .0000, .0000) 

[1.0, 1.5)  (.0015, .0115)  (.0135, .0415)  ( .0000, .0000) 

[1.5, 2.0)  (.0015, .0150)  (.0125, .0380)  ( .0000, .0000) 

[2. 0,2. 5)  (.0055, .0145)  ( .0115, .0320)  ( .0000, .0000) 

12.5,3.0)  (.0055, .0180)  ( .0105, .0260)  ( .0000, .0000) 

13.0,3.5)  (.0060, .0135)  ( .0095, .0210)  ( .0000, .0000) 

[3. 5, 4.0)  (.0055, .0095)  ( .0100, .0175)  ( .0000, .0000) 

14.0,4.5)  (.0055, .0110)  ( .0085, .0260)  ( .0000, .0000) 

[4. 5, 5.0)  (.0090, .0125)  ( .0070, .0115)  ( .0000, .0000) 

[4.5, 5.0)  (.0090, ,0125)  (.0070, .0115)  ( .0000, .0000) 


MINIMUM  N  (N  ) 
in 

TABLE:  2x3 


VECTOR  K 


>96 

>96 

>96 

>96 

62 

>96 

>96 

>96 

>96 

>96 

17 

>96 

>96 

33 

>96 

>96 

81 

>96 

>96 

>96 

>96 

>96 

25 

>96 

>96 

24 

>96 

>96 

36 

>96 

>96 

48 

>96 

>96 

13 

>96 

>96 

14 

>96 

>96 

<12 

>96 

>96 

30 

>96 

>96 

<12 

>96 

>96 

<12 

>96 

>96 

<12 

>96 

>96 

<12 

>96 

>96 

19 

>96 

36 

<12 

>96 

82 

<12 

>96 

96 

25 

>96 

89 

13 

>96 

>96 

30 

>96 

>96 

26 

>96 

>96 

>96 

>96 

>96 

<12 

>96 

>96 

<12 

>96 

45 

<12 

>96 

90 

22 

>96 

96 

20 

>96 

50 

<12 

>96 

64 

<12 

94 

80 

<12 

93 

75 

<12 

>96 

>96 

30 

>96 

>96 

40 

>96 

84 

30 

>96 

74 

<12 

>96 

69 

18 

>96 

50 

21 

>96 

50 

<12 

77 

48 

<12 

>96 

36 

20 

80 

48 

<12 

60 

45 

<12 

70 

35 

15 

>96 

>96 

32 

>96 

>96 

45 

>96 

>96 

61 

>96 

87 

24 

>96 

90 

26 

>96 

82 

32 

>96 

70 

<12 

84 

62 

18 

>96 

67 

24 

>96 

74 

<12 

58 

55 

<12 

72 

44 

22 

74 

49 

<12 

52 

38 

<12 

60 

36 

15 

65 

MINIMUM  N  (N  ) 
m 

TABLE:  2x4 


a 

=  . 

10 

a 

-  .1 

05 

a 

«  .i 

01 

ECTOR 

K 

P 

G 

K 

P 

G 

K 

P 

G 

1 

>96 

37 

>96 

>96 

76 

>96 

>96 

>96 

>96 

2 

>96 

21 

>96 

>96 

24 

>96 

>96 

94 

>96 

3 

>96 

22 

>96 

>96 

23 

>96 

>96 

80 

>96 

4 

>96 

50 

>96 

>96 

50 

>96 

>96 

<16 

>96 

5 

>96 

16 

>96 

>96 

<16 

>96 

>96 

25 

>96 

6 

>96 

<16 

>96 

>96 

<16 

>96 

>96 

32 

>96 

7 

>96 

<16 

>96 

>96 

<16 

>96 

>96 

<16 

>96 

8 

>96 

<16 

>96 

>96 

<16 

>96 

>96 

<16 

>96 

9 

>96 

40 

>96 

>96 

28 

>96 

>96 

>96 

>96 

10 

>96 

24 

>96 

>96 

16 

>96 

50 

20 

>96 

11 

>96 

18 

>96 

>96 

18 

>96 

40 

<16 

>96 

12 

>96 

16 

>96 

96 

<16 

>96 

82 

<16 

>96 

13 

>96 

40 

>96 

>96 

44 

>96 

78 

80 

>96 

14 

>96 

16 

>96 

>96 

70 

>96 

>96 

55 

>96 

15 

>96 

24 

>96 

>96 

28 

>96 

72 

40 

>96 

16 

74 

<16 

>96 

82 

30 

>96 

80 

30 

>96 

17 

>96 

47 

>96 

>96 

50 

>96 

>96 

68 

>96 

18 

>96 

35 

>96 

>96 

38 

>96 

85 

55 

>96 

19 

88 

19 

>96 

96 

28 

>96 

82 

37 

>96 

20 

58 

<16 

>96 

79 

21 

>96 

70 

23 

>96 

21 

70 

<16 

80 

80 

<16 

>96 

70 

24 

>96 

263 


MINIMUM  N  (N  ) 
m 

TABLE:  3x3 


a  ■ 

.10 

a  » 

.05 

a  ■ 

.01 

VECTOR 

K 

P 

K 

P 

K 

P 

1 

>108 

48 

>108 

60 

>108 

>108 

2 

>108 

3] 

>108 

54 

>108 

99 

3 

>108 

20 

>108 

27 

>108 

79 

4 

>108 

50 

>108 

32 

>108 

<18 

5 

>108 

26 

>108 

49 

>108 

91 

6 

>108 

<18 

>108 

<18 

>108 

52 

7 

>108 

<18 

>108 

22 

>108 

18 

8 

>108 

<18 

>108 

30 

>108 

30 

9 

>108 

<18 

>108 

<18 

>108 

46 

10 

>108 

<18 

>108 

<18 

>108 

<18 

11 

>108 

<18 

>108 

<18 

>108 

20 

12 

>108 

<18 

>108 

<18 

90 

27 

13 

>108 

<18 

>108 

<18 

>108 

<18 

14 

>108 

<18 

>108 

20 

78 

<18 

15 

>108 

<18 

>108 

<18 

>108 

<18 

16 

>108 

<18 

>108 

20 

>108 

20 

17 

90 

<18 

72 

<18 

74 

<18 

18 

80 

<18 

74 

<18 

72 

<18 

19 

80 

<18 

80 

23 

72 

25 

20 

80 

<18 

64 

<18 

67 

22 

21 

63 

<18 

51 

18 

51 

<18 

MINIMUM  N  (N  ) 
m 

TABLE:  2x5 


a  *  .10  a  ■  .05  g  ■  .Qi 

VECTOR  K  P  G  K  P  G  k  P  g 

1  >100  50  >100  >100  74  >100  >100  >100  >100 

2  >100  31  >100  >100  42  >100  >100  85  >100 

3  >100  24  >100  >100  44  >100  >100  54  >100 

4  >100  <20  >100  >100  55  >100  >100  55  >100 

5  >100  <20  >100  >100  <20  >100  >100  <20  >100 

6  >100  <20  >100  >100  50  >100  >100  <20  >100 

7  >100  55  >100  >100  58  >100  >100  46  >100 

8  >100  <20  >100  >100  43  >100  >100  56  >100 

9  >100  <20  >100  >100  40  >100  >100  33  >100 

10  >100  27  >100  >100  45  >100  >100  44  >100 

11  >100  27  >100  >100  31  >100  94  40  >100 

12  95  21  >100  97  24  >100  85  28  >100 

13  >100  39  >100  >100  60  >100  >100  52  >100 

14  >100  24  >100  >100  60  >100  96  40  >100 

15  >100  23  >100  >100  27  >100  86  32  >100 

16  79  <20  >100  85  25  >100  60  26  >100 


265 


MINIMUM  N  (N  ) 
m 

TABLE:  S2  x  2  x  2 


a  ■  .10  a  ■  .05  a  *  .01 


VECTOR 

K 

P 

G 

K. 

P 

G 

K 

P 

G 

1 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

2 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

3 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

4 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

5 

>104 

>104 

>104 

>104 

85 

>104 

>104 

>104 

>104 

6 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

7 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

>104 

8 

>104 

>104 

>104 

>104 

60 

>104 

>104 

>104 

>104 

9 

>104 

>104 

99 

>104 

48 

>104 

>104 

>104 

>104 

10 

>104 

93 

>104 

>104 

35 

>104 

>104 

>104 

>104 

11 

>104 

>104 

>104 

>104 

36 

>104 

>104 

>104 

>104 

12 

>104 

>104 

89 

>104 

42 

>104 

>104 

>104 

98 

13 

71 

56 

80 

99 

57 

91 

74 

>104 

85 

14 

99 

44 

35 

94 

36 

68 

81 

72 

91 

15 

74 

56 

51 

60 

49 

88 

64 

56 

79 

16 

54 

47 

40 

54 

42 

50 

59 

40 

50 

17 

58 

38 

38 

55 

40 

32 

51 

37 

36 

266 


MINIMUM  N  (N  ) 
m 

TABLE:  2x2x2 


a 

-  .10 

a 

-  .05 

a 

-  .01 

VECTOR 

K 

P 

G 

K 

P 

G 

K 

P 

G 

1 

>96 

>96 

>96 

>96 

>96 

>96 

>96 

>96 

>96 

2 

>96 

63 

>96 

>96 

>96 

>96 

>96 

>96 

>96 

3 

>96 

56 

>96 

>96 

73 

>96 

>96 

>96 

>96 

4 

>96 

37 

>96 

>96 

88 

>96 

>96 

>96 

>96 

5 

>96 

27 

>96 

>96 

31 

>96 

>96 

63 

>96 

6 

>96 

<16 

>96 

>96 

23 

>96 

>96 

46 

>96 

7 

>96 

<16 

>96 

>96 

<16 

>96 

>96 

<16 

>96 

8 

>96 

37 

>96 

>96 

34 

>96 

>96 

20 

>96 

9 

>96 

43 

>96 

>96 

56 

>96 

>96 

17 

>96 

10 

>96 

19 

>96 

>96 

35 

>96 

>96 

>96 

>96 

11 

>96 

<16 

>96 

>96 

<16 

>96 

>96 

41 

>96 

12 

>96 

<16 

>96 

>96 

<16 

>96 

>96 

42 

>96 

13 

>96 

<16 

>96 

>96 

<16 

>96 

83 

<16 

>96 

14 

>96 

<16 

>96 

>96 

<16 

>96 

>96 

<16 

>96 

15 

96 

<16 

>96 

>96 

<16 

>96 

68 

<16 

>96 

16 

>96 

<16 

>96 

72 

<16 

>96 

56 

<16 

>96 

17 

96 

<16 

>96 

>96 

<16 

>96 

>96 

<16 

>96 

18 

>96 

<16 

>96 

>96 

<16 

>96 

64 

<16 

>96 

19 

63 

<16 

>96 

63 

16 

>96 

61 

<16 

>96 

20 

66 

<16 

>96 

49 

<16 

>96 

63 

<16 

>96 

21 

63 

<16 

>96 

65 

<16 

>96 

45 

<16 

>96 

22 

56 

<16 

96 

48 

<16 

>96 

45 

<16 

96 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X2 
STATISTIC :GSK 
ALPHA:  . 10 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.06, .14, .24, .56) 

.0361 

80 

1 

1 

1 

1 

1 

1 

0 

0 

0 

0 

(.08, .12, .32, .48) 

.0259 

69 

2 

2 

1 

1 

1 

0 

0 

0 

0 

0 

( .10, . 10, .40, .40) 

.0225 

64 

2 

2 

2 

2 

0 

0 

0 

0 

0 

0 

(.09, .21, .21, .49) 

.0216 

57 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0 

(.12, .18, .28, .42) 

.0129 

36 

2 

2 

2 

2 

1 

1 

0 

0 

0 

0 

(.15, .15, .35, .35) 

.0100 

33 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.16, .24, .24, .36) 

.0051 

28 

3 

3 

3 

3 

1 

1 

0 

0 

0 

0 

(.20, .20, .30, .30) 

.0025 

26 

4 

4 

4 

2 

2 

0 

0 

0 

0 

0 

(.25,. 25,. 25,. 25) 

.0000 

25 

4 

4 

4 

4 

0 

0 

0 

0 

0 

0 

(.01, .09, .09, .81) 

.1056 

> 

96 

3 

3 

1 

1 

1 

1 

1 

1 

1 

1 

(.02, .08, .18, .72) 

.0769 

> 

96 

2 

2 

2 

1 

1 

1 

1 

1 

1 

0 

(.03, .07, .27, .63) 

.0564 

> 

96 

2 

2 

2 

2 

1 

1 

1 

1 

0 

0 

(.04, .16, .16, .64) 

.0531 

> 

96 

1 

1 

1 

1 

1 

1 

1 

0 

0 

0 

(.04, .06, .36, .54) 

.0441 

> 

9b 

2 

2 

2 

2 

2 

1 

1 

0 

0 

0 

(.05, .05, .45, .45) 

.0400 

> 

96 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

TABLE;  2X3 
STATISTIC :GSK 
ALPHA;  .10 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


P  VECTOR  MU2 

(.10, .10, .30, .10, .10, .30)  .0089 
(.06, .12, .12, .14, .28, .28)  .0070 
(.08, .12, .20, .12, .18, .30)  .0052 
(.10, .20, .20, .10, .20, .20)  .0022 
(.12, .12, .16, .18, .18, .24)  .0017 
(.17, .17, .17, .17, .17, .17)  .0000 


(.01, .01, .08, .09, .09, .72)  .0624 
(.03, .03, .24, .07, .07, .56)  .0360 
(.01, .03, .06, .09, .27, .54)  .0351 
(.02, .04, .14, .08, .16, .56)  .0334 
(.02, .02, .06, .18, .18, .54)  .0324 
(.05, .05, .40, .05, .05, .40)  .0272 
(.02, .04, .04, .18, .36, .36)  .0214 
(.02, .08, .10, .08, .32, .40)  .0198 
(.04, .08, .28, .06, .12, .42)  .0190 
(.03, .09, .18, .07, .21, .42)  .0167 
(.04, .06, .10, .16, .24, .40)  .0153 
(.06, .06, .18, .14, .14, .42)  .0148 
(.06, .06, .08, .24, .24, .32)  .0108 
(.05, .15, .30, .05, .15, .30)  .0106 
(.04, .16, .20, .06, .24, .30)  .0086 


NUMBER  OF  EXPECTED 


N(HXN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

84 

4 

4 

0 

0 

0 

0 

0 

0 

0 

0 

94 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0 

77 

3 

1 

1 

1 

0 

0 

0 

0 

0 

0 

58 

2 

2 

2 

2 

2 

0 

0 

0 

0 

0 

60 

3 

2 

2 

0 

0 

0 

0 

0 

0 

0 

52 

6 

6 

0 

0 

0 

0 

0 

0 

0 

0 

> 

96 

5 

5 

3 

2 

2 

2 

2 

2 

2 

2 

> 

96 

4 

4 

4 

4 

2 

2 

2 

2 

0 

0 

> 

96 

4 

4 

3 

3 

3 

2 

2 

2 

1 

1 

> 

96 

3 

3 

3 

2 

2 

2 

2 

1 

1 

0 

> 

96 

3 

3 

3 

3 

3 

2 

2 

2 

2 

0 

> 

96 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

> 

96 

3 

3 

3 

3 

3 

3 

3 

1 

1 

0 

> 

96 

4 

3 

3 

1 

1 

1 

1 

1 

1 

0 

> 

96 

3 

3 

3 

2 

2 

1 

1 

0 

0 

0 

> 

96 

3 

3 

2 

2 

1 

1 

1 

1 

0 

0 

> 

96 

3 

2 

2 

2 

2 

1 

1 

0 

0 

0 

> 

96 

2 

2 

2 

2 

2 

0 

0 

0 

0 

0 

> 

96 

3 

3 

3 

2 

2 

0 

0 

0 

0 

0 

> 

96 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

> 

96 

2 

2 

2 

2 

2 

1 

1 

0 

0 

0 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X4 
STATISTIC: GSK 
ALPHA:  .10 

NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.13,. 13,. 13,. 13,. 13,. 13,. 13,. 13) 

.0000 

80 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

( .01, .01, .01, .07, .09, .09, .09, .63) 

.0377 

> 

96 

7 

7 

4 

4 

3 

3 

3 

3 

3 

3 

(.03,. 03,. 03,. 21,. 07,. 07,. 07,. 49) 

.0221 

> 

96 

6 

6 

6 

6 

3 

3 

3 

3 

0 

0 

(.01,. 01, .03,. 05,. 09,. 09,. 27,. 45) 

.0213 

> 

96 

6 

6 

4 

4 

4 

4 

3 

3 

2 

2 

(.02,. 02, .04,. 12,. 08,. OS,. 16,. 48) 

.0201 

> 

96 

5 

5 

5 

3 

3 

3 

3 

2 

2 

0 

(.01,. 02,. 02,. 05,. 09,.  18,.  18,-. 45) 

.0192 

> 

96 

5 

5 

4 

4 

4 

4 

3 

3 

3 

1 

(.05,. 05,. 05,. 35,. 05,. 05,. 05,. 35) 

.0169 

> 

96 

6 

6 

6 

6 

6 

6 

0 

0 

0 

0 

(.02,. 02,. 08,. 08,. 08,. 08,. 32,. 32) 

.0133 

> 

96 

6 

6 

6 

2 

2 

2 

2 

2 

2 

0 

( .02, .02, .Q2, .04, . 18, . 18, . 18, .36) 

.0131 

> 

96 

4 

4 

4 

4 

4 

4 

4 

3 

3 

0 

(.04,. 04,. 08,. 24,. 06,. 06,. 12,. 36) 

.0117 

> 

96 

5 

5 

5 

4 

4 

2 

2 

0 

0 

0 

(.03,. 03,. 09,. 15,. 07,. 07,. 21,. 35) 

.0105 

> 

96 

5 

5 

4 

4 

2 

2 

2 

2 

0 

0 

(. 02,. 04,. 06,. U8,. 08,. 16,. 24,. 32) 

.0099 

> 

96 

5 

5 

5 

3 

3 

2 

2 

1 

1 

0 

(.03,. 06,. 06,. 15,. 07,. 14,. 14,. 35) 

.0090 

> 

96 

4 

4 

4 

4 

3 

1 

1 

1 

0 

(.05,. 05,. 15,. 25,. 05,. 05,. 15,. 25) 

.0069 

> 

96 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

(.04,. 04,. 16,. 16,. 06,. 06,. 24,. 24) 

.0065 

> 

96 

4 

4 

4 

4 

4 

2 

2 

0 

0 

0 

(.04,. 04,. 06,. 06,. 16,. 16,. 24,. 24) 

.0065 

> 

96 

4 

4 

4 

4 

4 

2 

2 

0 

0 

0 

(.05,. 10,. 10,. 25,. 05,. 10,. 10,. 25) 

.0056 

> 

96 

6 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.06,. 06,. 06,. 12,. 14,. 14,. 14,. 28) 

.0047 

> 

96 

3 

3 

3 

3 

3 

0 

0 

0 

0 

0 

(.04,. 08,. 12,. 16,. 06,. 12,. 18,. 24) 

.0039 

> 

96 

3 

3 

3 

2 

2 

1 

1 

0 

0 

0 

(.10,. 10,. 10,. 20,. 10,. 10,. 10,. 20) 

.0019 

> 

96 

6 

0 

0 

0 

0 

0 

0 

0 

0 

0 

(.08,. 08,. 12,. 12,. 12,. 12,. 18,. 18) 

.0013 

> 

96 

2 

2 

2 

0 

0 

0 

0 

0 

0 

0 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE;  2X2 
STATISTIC :GSK 
ALPHA:  .05 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.06, .14, .24, .56) 

.0361 

96 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0 

(.08, .12, .32, .48) 

.0259 

96 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

(.10, .10, .40, .40) 

.0225 

84 

2 

2 

0 

0 

0 

0 

0 

0 

0 

0 

(.09, .21, .21, .49) 

.0216 

70 

1 

1 

1 

1 

0 

0 

0 

0 

0 

0 

(.12, .18, .28, .42) 

.0129 

46 

2 

2 

1 

1 

1 

0 

0 

0 

0 

0 

(.15, .15, .35, .35) 

.0100 

44 

2 

2 

2 

2 

0 

0 

0 

0 

0 

0 

(.16, .24, .24, .36) 

.0051 

32 

3 

3 

3 

1 

1 

0 

0 

0 

0 

0 

(.20, .20, .30, .30) 

.0025 

30 

4 

2 

2 

2 

0 

0 

0 

0 

0 

0 

(.25, .25, .25, .25) 

.0000 

23 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0 

(.01, .09, .09, .81) 

.1056 

> 

96 

3 

3 

1 

1 

1 

1 

1 

1 

1 

1 

(.02, .08, .18, .72) 

.0769 

> 

96 

2 

2 

2 

1 

1 

1 

1 

1 

1 

0 

(.03, .07, .27, .63) 

.0564 

> 

96 

2 

2 

2 

2 

1 

1 

1 

1 

0 

0 

(.04, .16, .16, .64) 

.0531 

> 

96 

1 

1 

1 

1 

1 

1 

1 

0 

0 

0 

(.04, .06, .36, .54) 

.0441 

> 

96 

2 

2 

2 

2 

2 

1 

1 

0 

0 

0 

(.05, .05, .45, .45) 

.0400 

> 

96 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X3 
STATISTIC :GSK 
ALPHA:  .05 

NUMBER  OF  EXPECTED 

P  VECTOR  MU2  N(MIN)  VALUES  LESS  THAN 


10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

(.06, .12, .12,. 14, .28, .28) 

.0070 

93 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0 

(.10, .20, .20, .10, .20, .20) 

.0022 

72 

2 

2 

2 

0 

0 

0 

0 

0 

0 

0 

(.12, .12, .16, .18, .18, .24) 

.0017 

70 

2 

2 

0 

0 

0 

0 

0 

0 

0 

0 

(.17, .17, .17, .17, .17, .17) 

.0000 

60 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

(.01, .01, .08,. 09, .09, .72) 

•  u624 

> 

96 

5 

5 

3 

2 

2 

2 

2 

2 

2 

2 

(.U3,. 03, .24, .07, .07, .56) 

.0360 

> 

96 

4 

4 

4 

4 

2 

2 

2 

2 

0 

0 

(.01, .03, .06, .09, .27, .54) 

.0351 

> 

96 

4 

4 

3 

3 

3 

2 

2 

2 

1 

1 

(.02, .04, .14, .08, .16, .56) 

.0334 

> 

96 

3 

3 

3 

2 

2 

2 

2 

1 

1 

0 

(.02, .02, .06, .18, .18, .54) 

.0324 

> 

96 

3 

3 

3 

3 

3 

2 

2 

2 

2 

0 

(.05,. 05,. 40,. 05,. 05,. 40) 

.0272 

> 

96 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

(.02, .04, .04, .18, .36, .36) 

.0214 

> 

96 

3 

3 

3 

3 

3 

3 

3 

1 

1 

0 

(.02, .08, .10, .08, .32, .40) 

.0198 

> 

96 

4 

3 

3 

1 

1 

1 

1 

1 

1 

0 

(.04, .08, .28, .06, .12, .42) 

.0190 

> 

96 

3 

3 

3 

2 

2 

1 

1 

0 

0 

0 

( .03, .U9, . 18, .07, . 21 , . 42) 

.0167 

> 

96 

3 

3 

2 

2 

1 

1 

1 

1 

0 

0 

(.04, .06, .10, .16, .24, .40) 

.0153 

> 

96 

3 

2 

2 

2 

2 

1 

1 

0 

0 

0 

(.06, .06, .18, .14, .14, .42) 

.0148 

> 

96 

2 

2 

2 

2 

2 

0 

0 

0 

0 

0 

(.06, .06, .08, .24, .24, .32) 

.0108 

> 

96 

3 

3 

3 

2 

2 

0 

0 

0 

0 

0 

(.05, .15, .30, .05, .15, .30) 

.0106 

> 

96 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.10, .10, .30, .10,. 10, .30) 

.0089 

> 

96 

4 

0 

0 

0 

0 

0 

0 

0 

0 

0 

(.04, .16, .20, .06, .24, .30) 

.0086 

> 

96 

2 

2 

2 

2 

2 

1 

1 

0 

0 

0 

(.08, .12, .20, .12, .18, .30) 

.0052 

> 

96 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

TABLE:  2X4 
STATISTIC :GSK 
ALPHA:  .05 


P  VECTOR 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 
N(MIN)  VALUES  LESS  THAN 


10 

_9 

_8 

_7 

_6 

_5 

_4 

_3 

_2 

_1 

(.01,. 01,. 01,. 07,. 09,. 09,. 09 ,.63) 

.0377 

> 

96 

7 

7 

4 

4 

3 

3 

3 

3 

3 

3 

(.03, .03,. 03,. 21,. 07,. 07,.  07,.  49) 

.0221 

> 

96 

6 

6 

6 

6 

3 

3 

3 

3 

0 

0 

(.01,. 01,. 03,. 05,. 09,. 09,. 27,. 45) 

.0213 

> 

96 

6 

6 

4 

4 

4 

4 

3 

3 

2 

2 

(.02,. 02,. 04,. 12,. 08,. 08,. 16,. 48) 

.0201 

> 

96 

5 

5 

5 

3 

3 

•3 

3 

2 

2 

0 

(.01,. 02,. 02, .05,. 09,. IB,. IB,. 45) 

.0192 

> 

96 

5 

5 

4 

4 

4 

4 

3 

3 

3 

1 

(.05,. 05,. 05,. 35,. 05,. 05,. 05,. 35) 

.0169 

> 

96 

6 

6 

6 

6 

6 

6 

0 

0 

0 

0 

(.02,. 02,. 08,. 08,. 08,. 08,. 32,. 32) 

.0133 

> 

96 

6 

6 

6 

2 

2 

2 

2 

2 

2 

0 

(.02,. 02,. 02,. 04,. 18,. 18,. 18,. 36) 

.0131 

> 

96 

4 

4 

4 

4 

4 

4 

4 

3 

3 

0 

(.04,. 04,. 08,. 24,. 06,. 06,. 12,. 36) 

.0117 

> 

96 

5 

5 

5 

4 

4 

2 

2 

0 

0 

0 

(.03,. 03,. 09,. 15,. 07,. 07,. 21,. 35) 

.0105 

> 

96 

5 

5 

4 

4 

2 

2 

2 

2 

0 

0 

(.02,. 04,. 06,. 08,. 08,. 16,. 24,. 32) 

.0099 

> 

96 

5 

5 

5 

3 

3 

2 

2 

1 

1 

0 

(.03,. 06,. 06,. 15,. 07,. 14,. 14,. 35) 

.0090 

> 

96 

4 

4 

4 

4 

3 

1 

1 

1 

0 

0 

(.05,. 05,. 15,. 25,. 05,. 05,. 15,. 25) 

.0069 

> 

96 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

(.04,. 04,. 16, .16,. 06,. 06,. 24, .24) 

.0065 

> 

96 

4 

4 

4 

4 

4 

2 

2 

0 

0 

0 

(.04, .04,. 06 , • 06 , .16, .16, .24, .24) 

.0065 

> 

96 

4 

4 

4 

4 

4 

2 

2 

0 

0 

0 

(.05,. 10,. 10,. 25,. 05,. 10,. 10,. 25) 

.0056 

> 

96 

6 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.06,. 06,. 06,. 12,. 14,. 14,. 14,. 28) 

.0047 

> 

96 

3 

3 

3 

3 

3 

0 

0 

0 

0 

0 

(.04,. 08,. 12,. 16,. 06,. 12,. 18,. 24) 

.0039 

> 

96 

3 

3 

3 

2 

2 

1 

1 

0 

0 

0 

(.10,. 10,. 10,. 20,. 10,. 10,. 10,. 20) 

.0019 

> 

96 

6 

0 

0 

0 

0 

0 

0 

0 

0 

0 

(.08,. 08,. 12,. 12,. 12,. 12,. IS-. 18) 

.0013 

> 

96 

2 

2 

2 

0 

0 

0 

0 

0 

0 

0 

(.13,. 13,. 13,. 13,. 13,. 13,. 13,. 13) 

.0000 

> 

96 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X2 
STATISTIC :GSK 
ALPHA:  .01 

NUMBER  OF  EXPECTED 

P  VECTOR  .1U2  N(MIN)  VALUES  LESS  THAN 


10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

(.06, .14, .24, .56) 

.0361 

90 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0 

(.10, .10, .40, .40) 

.0225 

88 

2 

2 

0 

0 

0 

0 

0 

0 

0 

0 

(.09, .21, .21, .49) 

.0216 

78 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

(.12,. IB, .26, .4^) 

.0129 

60 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

(.15, .15, .35, .35) 

.0100 

56 

2 

2 

0 

0 

0 

0 

0 

0 

0 

0 

(.16, .24, .24, .36) 

.0051 

40 

3 

1 

1 

1 

0 

0 

0 

0 

0 

0 

(.20, .20, .30, .30) 

.0025 

37 

2 

2 

2 

0 

0 

0 

0 

0 

0 

0 

(.25, .25, .25, .25) 

.0000 

32 

4 

4 

0 

0 

0 

0 

0 

0 

0 

0 

(.01, .09, .09, .81) 

.1056 

> 

96 

3 

3 

1 

1 

1 

1 

1 

1 

1 

1 

(.02, .08, .18, .72) 

.0769 

> 

96 

2 

2 

2 

1 

1 

1 

1 

1 

1 

0 

( .03, .07, .27, .b3) 

.0564 

> 

9b 

2 

2 

2 

2 

1 

1 

1 

1 

0 

0 

( .04, . 16, . 16, .64) 

.0531 

> 

96 

1 

1 

1 

1 

1 

1 

1 

0 

0 

0 

(.04, .06, .36, .54) 

.0441 

> 

96 

2 

2 

2 

2 

2 

1 

1 

0 

0 

0 

(.05, .05, .45, .45) 

.0400 

> 

96 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.08, .12, .32, .48) 

.0259 

> 

96 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

(.08, .12, .32, .48) 

.0259 

> 

96 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 
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TABLE:  2X3 
STATISTIC :GSK 
ALPHA:  .01 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.08, .12, .20, .12, .18, .30) 

.0052 

80 

3 

1 

1 

1 

0 

0 

0 

0 

0 

0 

(.10, .20, .20, .10, .20, .20) 

.0022 

74 

2 

2 

2 

0 

0 

0 

0 

0 

0 

0 

(.17, .17, .17, .17, .17, .17) 

.0000 

65 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

(.01, .01, .08, .09, .09, .72) 

.0624 

> 

96 

5 

5 

3 

2 

2 

2 

2 

2 

2 

2 

(.03, .03, .24, .07, .07, .56) 

.0360 

> 

96 

4 

4 

4 

4 

2 

2 

2 

2 

0 

0 

(.01, .03, .06, .09, .27, .54) 

.0351 

> 

96 

4 

4 

3 

3 

3 

2 

2 

2 

1 

1 

(.02, .04, .14, .08, .16, .56) 

.0334 

> 

96 

3 

3 

3 

2 

2 

2 

2 

1 

1 

0 

(.02,. 02,. 06,. 18,. 18,. 54) 

.0324 

> 

96 

3 

3 

3 

3 

3 

2 

2 

2 

2 

0 

(.05, .05, .40, .05, .05, .40) 

.0272 

> 

96 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

( .02, .04, .04, . 18, .36, .36) 

.0214 

> 

96 

3 

3 

3 

3 

3 

3 

3 

1 

1 

0 

(.02, .08, .10, .08, .32, .40) 

.0198 

> 

96 

4 

3 

3 

1 

1 

1 

1 

1 

1 

0 

(.04, .08, .28, .06, .12, .42) 

.0190 

> 

96 

3 

3 

3 

2 

2 

1 

1 

0 

0 

0 

(.03, .09, .18, .07, .21, .42) 

.0167 

> 

96 

3 

3 

2 

2 

1 

1 

1 

1 

0 

0 

(.04,. 06,. 10,. 16,. 24,. 40) 

.0153 

> 

96 

3 

2 

2 

2 

2 

1 

1 

0 

0 

0 

(.06, .06, .18, .14, .14, .42) 

.0148 

> 

96 

2 

2 

2 

2 

2 

0 

0 

0 

0 

0 

(.06,. 06,. 08,. 24,. 24,. 32) 

.0108 

> 

96 

3 

3 

3 

2 

2 

0 

0 

0 

0 

0 

(.05, .15, .30,. 05, .15, .30) 

.0106 

> 

96 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.10, .10, .30, .10, .10, .30) 

.0089 

> 

96 

4 

0 

0 

0 

0 

0 

0 

0 

0 

0 

(.04,. 16, .20, .06, .24, .30) 

.0086 

> 

96 

2 

2 

2 

2 

2 

1 

1 

0 

0 

0 

(.06, .12, .12, .14, .28, .28) 

.0070 

> 

96 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0 

(.12,. 12, .16, .18, .18, .24) 

.0017 

> 

96 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

(.12, .12, .16, .18, .18, .24) 

.0017 

> 

96 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

TABLE:  2X4 
STATISTIC :GSK 
ALPHA:  .01 


P  VECTOR 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 
N(MIN)  VALUES  LESS  THAN 
10  987654321 


(.01, 

(.03, 

(.01, 

(.02, 

(.01, 

(.05, 

(.02, 

(.02, 

(.04, 

(.03, 

(.02, 

(.03, 

(.05, 

(.04, 

(.04, 

(.05, 

(.06, 

(.04, 

(.10, 

(.08, 

(.13, 

(.13, 


,.07, 

.09, 

.09, 

.09, 

.63) 

,.21, 

.07, 

.07, 

.07, 

AO'S 
•  ^  / 

,.05, 

.09, 

.09, 

.27, 

.45) 

,.12, 

.08, 

.08, 

.16, 

.48) 

,.05, 

.09, 

.18, 

.18, 

.45) 

,.35, 

.05, 

.05, 

.05, 

.35) 

,  *08, 

.08, 

.08, 

.32, 

.32) 

,-04, 

.18, 

.18, 

.18, 

.36) 

,-24, 

.06, 

.06, 

.12, 

.36) 

,.15, 

.  U7 , 

.07, 

•21, 

• » ->) 

,  *08, 

.08, 

.16, 

.24, 

.32) 

,.15, 

.07, 

.14, 

•14, 

.35) 

,.25, 

.05, 

.05, 

•15, 

.25) 

,  •  16, 

.06, 

.06, 

•24, 

.24) 

,  *06, 

.16, 

.16, 

•24, 

.24) 

,.25, 

.05, 

.10, 

.10, 

.25) 

,.12, 

•  14, 

.14, 

.14, 

.28) 

,.16, 

.06, 

.12, 

.18, 

.24) 

,.20, 

.10, 

.10, 

.10, 

.20) 

,.12, 

.12, 

•12, 

.18, 

.18) 

,.13, 

•13, 

•13, 

.13, 

.13) 

.0377 

.0221 

.0213 

.0201 

.0192 

.0169 

.0133 

.0131 

.0117 

.0105 

.0099 

.0090 

.0069 

.0065 

.0065 

.0056 

.0047 

.0039 

.0019 

.0013 

.0000 

.0000 


7  7  4 
6  6  6 
6  6  4 
5  5  5 

5  5  4 

6  6  6 
6  6  6 

4  4  4 

5  5  5 
5  5  4 

5  5  5 
4  4  4 
4  4  4 
4  4  4 
4  4  4 

6  2  2 
3  3  3 
3  3  3 
6  0  0 
2  2  2 
0  0  0 
0  0  0 


4  3  3 
6  3  3 
4  4  4 

3  3  3 

4  4  4 
6  6  6 
2  2  2 
4  4  4 
4  4  2 
4  2  2 

3  3  2 

4  3  1 
4  4  4 
4  4  2 
4  4  2 
2  2  2 
3  3  0 
2  2  1 
0  0  0 
0  0  0 
0  0  0 
0  0  0 


3  3  3  3 
3  3  0  0 
3  3  2  2 
3  2  2  0 

3  3  3  1 
0  0  0  0 
2  2  2  0 

4  3  3  0 
2  0  0  0 
2  2  0  0 
2  110 
110  0 
0  0  0  0 
2  0  0  0 
2  0  0  0 
0  0  0  0 
0  0  0  0 
10  0  0 
0  0  0  0 
0  0  0  0 
0  0  0  0 
0  0  0  0 


.9 


9 


Q 


B 


S' 


278 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


TABLE:  S 2X2X2 
STATISTIC :GSK 
ALPHA:  .10 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

( .032, .048, .048, .072, . 128, . 192, . 192, .288) 

.0074 

99 

4 

4 

4 

3 

3 

3 

1 

0 

0 

0 

( .045, .045, .105, . 105, .105, . 105, .245, .245) 

.0054 

89 

6 

2 

2 

2 

2 

2 

0 

0 

0 

0 

( .048, .072, .072, . 108, . 112, . 168, . 168, .252) 

.0040 

80 

5 

5 

3 

3 

3 

1 

1 

0 

0 

0 

(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 

.0025 

65 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

(.064, .096, .096, .144, .096, .144, .144, .216) 

.0020 

51 

7 

7 

7 

4 

4 

4 

1 

0 

0 

0 

( . 100, . 100, . 100, . 100, . 150, . 150, . 150, . 150) 

.0006 

40 

8 

8 

8 

8 

4 

4 

0 

0 

0 

0 

( . 125, . 125, . 125, . 125, . 125, . 125, . 125, . 125) 

.0000 

38 

8 

8 

8 

8 

8 

8 

0 

0 

0 

0 

( .008, .012, .032, .048, .072, . 108, .288, .432) 

.0206 

>104 

5 

5 

5 

4 

4 

4 

3 

2 

2 

1 

(.009,. 021,. 021,. 049,. 081,. 189,. 189,. 441) 

.0189 

>104 

5 

5 

4 

4 

4 

3 

3 

3 

1 

1 

( .016, .024, .064, .096, .064, .096, .256, .384) 

.0144 

>104 

6 

4 

4 

4 

2 

2 

2 

2 

1 

0 

( .015, .015, .035, .035, . 135, . 135, .315, .315) 

.0141 

>104 

4 

4 

4 

4 

4 

4 

4 

2 

2 

0 

( . 018 , . 042 , . 042 , . 098 , . 072 , . 168 , . 16~ , . 392 ) 

.0130 

>104 

4 

4 

4 

3 

3 

3 

1 

1 

1 

0 

(.016, .024,. 024,. 036,. 144,. 216,. 216,. 324) 

.0121 

>104 

4 

4 

4 

4 

4 

4 

4 

3 

1 

0 

( . 025 , . 025 , . 025 , . 025 , . 225 , . 225 , . 225 , . 2 25 ) 

.0100 

>104 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

(.030,. 030,. 070,. 070,. 120,. 120,. 280,. 280) 

.0090 

>104 

4 

4 

4 

2 

2 

2 

2 

0 

0 

0 

(.027,. 063,. 063,. 147,. 063,. 147,. 147,. 343) 

.0088 

>104 

4 

4 

4 

4 

1 

1 

1 

1 

0 

0 

( .050, .050,. 050, .050, .200, .200, .200, .200) 

.0056 

>104 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0 

A'.Jhtea2j2Um''*:Xl4b--£m  G  l**i!X*  *-*  V,  ♦.>  •.jk.'>*4A.,-».. 
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TABLE:  2X2X2 
STATISTIC :GSK 
ALPHA:  .10 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

( . 125, . 125, . 125, . 125, . 125, . 125, . 125, . 125) 

.0000 

96 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

(.001,. 009,. 009,. 081,. 009,. 081,. 081,. 729) 

.0533 

> 

96 

7 

7 

7 

4 

4 

4 

4 

4 

4 

4 

(.003,. 007,. 027,. 063,. 027,. 063,. 243,. 567) 

.0331 

> 

96 

6 

6 

6 

6 

4 

4 

4 

4 

2 

2 

( .004, .016, .016, .064, .036, . 144, . 144, .576) 

.0318 

> 

96 

5 

5 

5 

5 

4 

4 

4 

3 

3 

1 

( .005, .005, .045, .045, .045, .045, .405, .405) 

.0264 

> 

96 

6 

6 

6 

6 

6 

6 

2 

2 

2 

2 

(.008,. 032,. 032,. 128,. 032,. 128,. 128,. 512) 

.0237 

> 

96 

4 

4 

4 

4 

4 

4 

4 

1 

1 

1 

( .008, .012, .032, .048, .072, . 108, .288, .432) 

.0206 

> 

96 

5 

5 

5 

5 

4 

4 

3 

2 

2 

1 

(.009,. 021,. 021,. 049,. 081,. 189,. 189,. 441) 

.0189 

> 

96 

5 

5 

5 

4 

4 

4 

3 

3 

1 

1 

( .016, .024, .064, .096, .064, .096, .256, .384) 

.0144 

> 

96 

6 

4 

4 

4 

2 

2 

2 

2 

1 

0 

(.015,. 015,. 035,. 035,. 135,. 135,. 315,. 315) 

.0141 

> 

96 

4 

4 

4 

4 

4 

4 

4 

2 

2 

0 

(.018,. 042,. 042,. 098,. 072,. 168,. 168,. 392) 

.0130 

> 

96 

5 

4 

4 

4 

3 

3 

1 

1 

1 

0 

(.016,. 024,. 024,. 036,. 144,. 216,. 216,. 324) 

.0121 

> 

96 

4 

4 

4 

4 

4 

4 

4 

3 

1 

0 

( .025, .025, .025, .025, .225, .225, .225, .225) 

.0100 

> 

96 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

( .030, .030, .070, .070, . 120, . 120, .280, .280) 

.0090 

> 

96 

4 

4 

4 

4 

2 

2 

2 

2 

0 

0 

(.027,. 063,. 063,. 147,. 063,. 147,. 147,. 343) 

.0088 

> 

96 

4 

4 

4 

4 

1 

1 

1 

1 

0 

0 

(.032,. 048,. 048,. 072,. 128,. 192,. 192,. 288) 

.0074 

> 

96 

4 

4 

4 

4 

3 

3 

1 

0 

0 

0 

( . 050 , . 050 , . 050 , . 050 , . 200 , . 200 , . 200 , . 200 ) 

.0056 

> 

96 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

(.045,. 045,. 105,. 105,. 105,. 105,. 245,. 245) 

.0054 

> 

96 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.048,. 072,. 072,. 108,. 112,. 168,. 168,. 252) 

.0040 

> 

96 

3 

3 

3 

3 

1 

1 

0 

0 

0 

0 

(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 

.0025 

> 

96 

4 

4 

4 

0 

0 

0 

0 

0 

0 

0 

(.064,. 096,. 096,. 144,. 096,. 144,. 144,. 216) 

.0020 

> 

96 

4 

1 

1 

1 

0 

0 

0 

0 

0 

0 

(.100,. 100,. 100,. 100,. 150,. 150,. 150,. 150) 

.0006 

> 

96 

4 

0 

0 

0 

0 

0 

0 

0 

0 

0 

TABLE:  S 2X2X2 
STATISTIC :GSK 
ALPHA:  .05 

P  VECTOR 

( .048, .072, .072, . 108, . 112, . 168, . 168, . 252) 
(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 
( .064, .096, .096, . 144, .096, . 144, . 144, .216) 
(.100,. 100,. 100,. 100,. 150,. 150,. 150,. 150) 
(.125,. 125,. 125,. 125,. 125,. 125,. 125,. 125) 

( .008, .012, .032, .048, .072,. 108, .288, .432) 
( .009, .021, .021, .049, .081, . 189, . 189, .441) 
( .016, .024, .064, .096, .064, .096, .256, .384) 
(.015,. 015,. 035,. 035,. 135,. 135,. 315,. 315) 
( .018, .042, .042, .098, .072, . 168, . 168, .392) 
(.016,. 024,. 024,. 036,. 144,. 216,. 216,. 324) 
( .025, .025, .025, .025, .225, .225, .225, .225) 
( .030, .030, .070, .070, . 120, . 120, .280, .280) 
( . 027 , . 063 , . 063 , . 147 , . 063 , . 147 , . 147 , . 343 ) 
( .032, .048, .048, .072, . 128, . 192, . 192, .288) 
( .050, .050, .050, .050, .200, .200, .200, .200) 
( . 045 , . 045 , . 105 , . 105 , . 105 , . 105 , . 245 , . 245 ) 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0040 

91 

4 

3 

3 

3 

1 

1 

0 

0 

0 

0 

0025 

68 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0 

0020 

88 

4 

4 

1 

1 

1 

0 

0 

0 

0 

0 

0006 

50 

8 

8 

8 

4 

4 

0 

0 

0 

0 

0 

0000 

32 

8 

8 

8 

8 

8 

8 

0 

0 

0 

0 

0206 

>104 

5 

5 

5 

4 

4 

4 

3 

2 

2 

1 

0189 

>104 

5 

5 

4 

4 

4 

3 

3 

3 

1 

1 

0144 

>104 

6 

4 

4 

4 

2 

2 

2 

2 

1 

0 

0141 

>104 

4 

4 

4 

4 

4 

4 

4 

2 

2 

0 

0130 

>104 

4 

4 

4 

3 

3 

3 

1 

1 

1 

0 

0121 

>104 

4 

4 

4 

4 

4 

4 

4 

3 

1 

0 

0100 

>104 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

0090 

>104 

4 

4 

4 

2 

2 

2 

2 

0 

0 

0 

0088 

>104 

4 

4 

4 

4 

1 

1 

1 

1 

0 

0 

0074 

>104 

4 

4 

4 

3 

3 

3 

1 

0 

0 

0 

0056 

>104 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0 

0054 

>104 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

tgXffWf 
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TABLE:  2X2X2 
STATISTIC :GSK 
ALPHA:  .05 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


'f«7 


© 


& 


P  VECTOR 

MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

_9_ 

_8_ 

_7_ 

_b_ 

_5_ 

_4_ 

J_ 

_2_ 

_1 

(. 001,. 009,. 009,. 081,. 009,. 081,. OBI,. 729) 

.0533 

> 

96 

7 

7 

7 

4 

4 

4 

4 

4 

4 

4 

( .003, .007, .027, .063, .027, .063, .243, .567) 

.0331 

> 

96 

6 

6 

6 

6 

4 

4 

4 

4 

2 

2 

( .004, .016, .016, .064, .036, . 144, . 144, .576) 

.0318 

> 

96 

5 

5 

5 

5 

4 

4 

4 

3 

3 

1 

( .005, .005, .045, .045, .045, .045, .405, .405) 

.0264 

> 

96 

6 

6 

6 

6 

6 

6 

2 

2 

2 

2 

(. 008,. 032,. 032,. 128,. 032,. 126,. 128,. 512) 

.0237 

> 

96 

4 

4 

4 

4 

4 

4 

4 

1 

1 

1 

( .006, .012, .032, .048, .072, . 108, .288, .432) 

.0206 

> 

96 

5 

5 

5 

5 

4 

4 

3 

2 

2 

1 

(.009,. 021,. 021,. 049, .081,. 189,. 189,. 441) 

.0189 

> 

96 

5 

5 

5 

4 

4 

4 

3 

3 

1 

1 

( .016 , . 024 , . 064 , . 096, . 064 , . 096 , . 256 , . 384) 

.0144 

> 

96 

6 

4 

4 

4 

2 

2 

2 

2 

1 

0 

(.015,. 015,. 035,. 035,. 135,. 135,. 315,. 315) 

.0141 

> 

96 

4 

4 

4 

4 

4 

4 

4 

2 

2 

0 

( .018, .042, .042, .098, .072, . 168, . 168, .392) 

.0130 

> 

96 

5 

4 

4 

4 

3 

3 

1 

1 

1 

0 

(.016,. 024,. 024,. 036,. 144,. 216,. 216,. 324) 

.0121 

> 

96 

4 

4 

4 

4 

4 

4 

4 

3 

1 

0 

( . 025 , . 025 , . 025 , . 025 , . 225 , . 225 , . 225 , . 225 ) 

.0100 

> 

96 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

( .030, .030, .070, .070, .120, . 120, .280, .280) 

.0090 

> 

96 

4 

4 

4 

4 

2 

2 

2 

2 

0 

0 

( . 027 , . 063, . 063, . 147 , . 063 , . 147 , . 147 , . 343) 

.0088 

> 

96 

4 

4 

4 

4 

1 

1 

1 

1 

0 

0 

(.032,. 048,. 048,. 072,. 128,. 192,. 192,. 288) 

.0074 

> 

96 

4 

4 

4 

4 

3 

3 

1 

0 

0 

0 

( . 050 , . 050 , . 050 , . 050 , . 200 , . 200 , . 200 , . 200 ) 

.0056 

> 

96 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

(.045,. 045,. 105,. 105,. 105,. 105,. 245,. 245) 

.0054 

> 

96 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.048,. 072,. 072,. 108,. 112,. 168,. 168,. 252) 

.0040 

> 

96 

3 

3 

3 

3 

1 

1 

0 

0 

0 

0 

(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 

.0025 

> 

96 

4 

4 

4 

0 

0 

0 

0 

0 

0 

0 

(.064,. 096,. 096,. 144,. 096,. 144,. 144,. 216) 

.0020 

> 

96 

4 

1 

1 

1 

0 

0 

0 

0 

0 

0 

( . 100, . 100, . 100, . 100, . 150, . 150, . 150, . 150) 

.0006 

> 

96 

4 

0 

0 

0 

0 

0 

0 

0 

0 

0 

(.125,. 125,. 125,. 125,. 125,. 125,. 125,. 125) 

.0000 

> 

96 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

282 
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TABLE:  S2X2X2 
STATISTIC :GSK 
ALPHA:  .01 

P  VECTOR 

( . 045 , . 045 , . 105 , . 105, . 105, . 105 , . 245 , . 245 ) 
(.048,. 072,. 072,. 108,. 112,. 168,. 168,. 252) 
(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 
( .064, .096, .096, . 144, .096, . 144, . 144, .216) 
(.100,. 100,. 100,. 100,. 150,. 150,. 150,. 150) 
( .125, . 125, . 125, . 125, . 125, . 125, . 125, . 125) 

(.008,. 012,. 032,. 048,. 072,. 108,. 288,. 432) 
( .  009 , .  02 1 , . 021 , . 049 , . 081 , . 189 , . 189 , . 441 ) 
( .016, .024, .064, .096, .064, .096, .256, .384) 
(.015,. 015,. 035,. 035,. 135,. 135,. 315,. 315) 
(.018,. 042,. 042,. 098,. 072,. 168,. 168,. 392) 
( .016, .024, .024, .036, .144, .216, .216, .324) 
( .025, .025, .025, .025, .225, .225, .225, .225) 
( .030, .030, .070, .070, . 120, . 120, .280, .280) 
(.027,. 063,. 063,. 147,. 063,. 147,. 147,. 343) 
( .032, .048, .048, .072, . 128, .  192, . 192, .288) 
( . 050 , . 050 , . 050 , . 050 , . 200 , . 200 , . 200 , . 200 ) 
( .050, .050, .050, .050, .200, .200, .200, .200) 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0054 

98 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

0040 

85 

5 

3 

3 

3 

1 

1 

0 

0 

0 

0 

0025 

91 

4 

4 

4 

4 

0 

0 

0 

0 

0 

0 

0020 

79 

4 

4 

4 

1 

1 

0 

0 

0 

0 

0 

0006 

50 

8 

8 

8 

4 

4 

0 

0 

0 

0 

0 

0000 

36 

8 

8 

8 

8 

8 

8 

0 

0 

0 

0 

0206 

>104 

5 

5 

5 

4 

4 

4 

3 

2 

2 

1 

0189 

>104 

5 

5 

4 

4 

4 

3 

3 

3 

1 

1 

0144 

>104 

6 

4 

4 

4 

2 

2 

2 

2 

1 

0 

0141 

>104 

4 

4 

4 

4 

4 

4 

4 

2 

2 

0 

0130 

>104 

4 

4 

4 

3 

3 

3 

1 

1 

1 

0 

0121 

>104 

4 

4 

4 

4 

4 

4 

4 

3 

1 

0 

0100 

>104 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

0090 

>104 

4 

4 

4 

2 

2 

2 

2 

0 

0 

0 

0088 

>104 

4 

4 

4 

4 

1 

1 

1 

1 

0 

0 

0074 

>104 

4 

4 

4 

3 

3 

3 

1 

0 

0 

0 

0056 

>104 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0 

0056 

>104 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0 

i 


V 
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TABLE:  2X2X2 
STATISTIC :GSK 
ALPHA:  .01 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

(.125,. 125,. 125,. 125,. 125,. 125,. 125,. 125) 

.0000 

96 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

(.001,. 009,. 009,. 081,. 009,. 081,. 081,. 729) 

.0533 

> 

96 

7 

7 

7 

4 

4 

4 

4 

4 

4 

4 

(.003, .007, .027, .063, .027, .063, .243, .567) 

.0331 

> 

96 

6 

6 

6 

6 

4 

4 

4 

4 

2 

2 

( .004, .016, .016, .064, .036, . 144, . 144, .576) 

.0318 

> 

96 

5 

5 

5 

5 

4 

4 

4 

3 

3 

1 

(.005,. 005,. 045,. 045,. 045,. 045,. 405,. 405) 

.0264 

> 

96 

6 

6 

6 

6 

6 

6 

2 

2 

2 

2 

( .008, .032, .032, . 128, .032, . 128, .128, .512) 

.0237 

> 

96 

4 

4 

4 

4 

4 

4 

4 

1 

1 

1 

(.008, .012, .032, .048, .072, . 108, .288, .432) 

.0206 

> 

96 

5 

5 

5 

5 

4 

4 

3 

2 

2 

1 

(.009,. 021,. 021,. 049,. 081,. 189,. 189,. 441) 

.0189 

> 

96 

5 

5 

5 

4 

4 

4 

3 

3 

1 

1 

( .016, .024, .064, .096, .064, .096, .256, .384) 

.0144 

s 

4 

96 

6 

4 

4 

4 

2 

2 

2 

2 

1 

0 

(.015,. 015,. 035,. 035,. 135,. 135,. 315,. 315) 

.0141 

> 

96 

4 

4 

4 

4 

4 

4 

4 

2 

2 

0 

C.018, .042, .042, .098, .072, .168, . 168, .392) 

.0130 

> 

96 

5 

4 

4 

4 

3 

3 

1 

1 

1 

0 

( .016, .024, .024, .036, . 144, .216, . 216, . 324) 

.0121 

> 

96 

4 

4 

4 

4 

4 

4 

4 

3 

1 

0 

( .025, .025, .025, .025, .225, .225, .225, .225) 

.0100 

> 

96 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

( .030, .030, .070, .070, . 120, . 120, .280, .280) 

.0090 

> 

96 

4 

4 

4 

4 

2 

2 

2 

2 

0 

0 

(.027,. 063,. 063,. 147,. 063,. 147,. 147,. 343) 

.0088 

> 

96 

4 

4 

4 

4 

1 

1 

1 

1 

0 

0 

(.032,. 048,. 048,. 072,. 128,. 192,. 192,. 288) 

.0074 

> 

96 

4 

4 

4 

4 

3 

3 

1 

0 

0 

0 

( . 050 , . 050 , . 050, . 050 , . 200 , . 200 , . 200 , . 200 ) 

.0056 

> 

96 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

(.045,. 045,. 105,. 105,. 105,. 105,. 245,. 245) 

.0054 

> 

96 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.048,. 072,. 072,. 108,. 112,. 168,. 168,. 252) 

.0040 

> 

96 

3 

3 

3 

3 

1 

1 

0 

0 

0 

0 

(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 

.0025 

> 

96 

4 

4 

4 

0 

0 

0 

0 

0 

0 

0 

( .064, .096, .096, . 144, .096, . 144, . 144, .216) 

.0020 

> 

96 

4 

1 

1 

1 

0 

0 

0 

0 

0 

0 

( . 100, . 100, . 100, . 100, . 150, . 150, . 150, . 150) 

.0006 

> 

96 

4 

0 

0 

0 

0 

0 

0 

0 

0 

0 

(.100,. 100,. 100,. 100,. 150,. 150,. 150,. 150) 

.0006 

> 

96 

4 

0 

0 

0 

0 

0 

0 

0 

0 

0 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X2 
STATISTIC  jKULLBACK 
ALPHA:  .10 


NUMBER  OF  EXPECTED 

P  VECTOR  MU2  N(MIN)  VALUES  LESS  THAN 


10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

(.04, .16, .16,. 64) 

.0531 

86 

1 

1 

1 

1 

1 

1 

1 

0 

0 

0 

(.04,. 06,. 36,. 54) 

.0441 

96 

2 

2 

2 

2 

2 

1 

1 

0 

0 

0 

(.05,. 05,. 45,. 45) 

.0400 

88 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.06, .14, .24, .56) 

.0361 

62 

2 

2 

1 

1 

1 

1 

1 

0 

0 

0 

(.08, .12, .32, .46) 

.0259 

48 

2 

2 

2 

2 

2 

1 

1 

0 

0 

0 

(.10, .10, .40, .40) 

.0225 

45 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.09, .21, .21, .49) 

.0216 

46 

3 

1 

1 

1 

1 

1 

0 

0 

0 

0 

(.12, .18, .28, .42) 

.0129 

35 

3 

2 

2 

2 

1 

1 

0 

0 

0 

0 

(.15, .15, .35, .35) 

.0100 

35 

2 

2 

2 

2 

2 

0 

0 

0 

0 

0 

(.16, .24, .24, .36) 

.0051 

35 

3 

3 

1 

1 

1 

0 

0 

0 

0 

0 

(.20,. 20,. 30,. 30) 

.0025 

36 

2 

2 

2 

0 

0 

0 

0 

0 

0 

0 

(.25, .25, .25, .25) 

.0000 

37 

4 

0 

0 

0. 

0 

0 

0 

0 

0 

0 

(.01, .09, .09, .81) 

.1056 

> 

96 

3 

3 

1 

1 

1 

1 

1 

1 

1 

1 

(.02, .08, .18, .72) 

.0769 

> 

96 

2 

2 

2 

1 

1 

1 

1 

1 

1 

0 

(.03, .07, .27, .63) 

.0564 

> 

96 

2 

2 

2 

2 

1 

1 

1 

1 

0 

0 

I  * 


09 
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CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


TABLE:  2X3 
STATISTIC iKULLBACK 
ALPHA:  .10 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10  9  8  7 

6 

3 

4 

3  2 

1 

( .08, *  06  , . 18, . 14, . 14, ,42) 

.0148 

90 

2  2  2  2  2 

0  0  0  0 

0 

( i06,»O6, . 08, . 24, . 24, . 32) 

.0108 

82 

3  3  3  3 

2 

2  0 

0  0 

0 

(.05,. 15, .30, .05, . 15, . 30) 

.0106 

87 

2  2  2  2 

2 

2 

0 

0  0 

0 

( •  10, .10, • 30, .10, . 10, .30) 

.0089 

70 

4  4  4  0 

0 

0 

0 

0  0 

0 

( .04,. 16, .20, .06, . 24,.  30) 

.0086 

74 

2  2  2  2 

2 

2 

1 

1  0 

0 

( .06,.  12,.  12,  *  14, <0 28 28)  . 

.0070 

64 

4  4  3  1 

1 

1 

1 

0  0 

0 

( *08, . 12, .20, .12, « 18,. 36) 

*6652 

50 

4  3  3  3 

1 

1 

0  0  0 

0 

(  *10,  *: 20,  .20, .  10, «  20,  .20)  • 

.0022 

74 

2  2  2  0  0  0  0 

0  0 

0 

( . 12, . 12, .16,. 18, * 18, .24) 

.0017 

48 

5  5  3  2  2 

0  0 

0  0 

0 

C .17, . 17 , . 17 , . 17, .17, . 17) 

.0000 

49 

6  6  0  0 

0 

0 

0 

0  0 

0 

i|l 

( .01, .01 , .08, .09, . 09, .72) 

*0624 

>  96 

5  5  3  2 

2 

2 

2 

2  2 

2 

( .03, .03,. 24,. 07, .07,. 56) 

.0360 

>  96 

4  4  4  4 

2 

2 

2 

2  0 

0 

( .01,  .03;,  .06,  . 09,  *27 ,  .  54) 

.0551 

>  96 

4  4  3  3 

3 

2 

2 

2  1 

1 

( .02,  *04, « 14,  *  08,  .16,.567 

.0334 

>  96 

3  3  3  2 

2 

2 

2 

1  1 

0 

(.02, .02,. 06, . 18, . 18, .54) 

.0524 

>  96 

3  3  3  3 

3 

2 

2 

2  2 

0 

( . 05, ,05, ,40, • 05, .05, . 40) 

.0272 

>  96 

4  4  4  4 

4 

4 

0  0  0 

0 

( .02, .04, .04, , 18,. 36, .36) 

.0214 

>  96 

3  3  3  3 

3 

3 

3 

1  1 

0 

( .02, . 08, . 10, .08, . 32 , *40) 

.0198 

>  96 

4  3  3  1 

1 

1 

1 

1  1 

0 

( . 04, .08, .28,. 06,. 12, .42) 

.0190 

>  96 

3  3  3  2 

2 

1 

1 

0  0 

0 

(.03,. 09,. 18, .07, *21, .42) 

.0167 

>  96 

3  3  2  2 

1 

1 

1 

1  0 

0 

(.04,. 06,. 10,. 16,. 24,. 40) 

.0153 

>  96 

3  2  2  2 

2 

1 

1 

0  0 

0 

TABLE:  2X4 
STATISTIC : KULLBACK 
ALPHA:  .10 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


P  VECTOR 


( 

.05, 

.10, 

.10, 

.  25 , 

.05, 

.10, 

.10, 

.25) 

(. 

.10, 

.10, 

.10, 

.20, 

.10, 

.10, 

.10, 

.20) 

( 

.08, 

.08, 

.12, 

•12, 

•12, 

.12, 

.18, 

.18) 

(. 

.13, 

•13, 

.13, 

.13, 

•13, 

.13, 

.13, 

.13) 

(. 

.01, 

.01, 

.01, 

.07, 

.09, 

.09, 

.09, 

.63) 

<■ 

.03, 

.03,. 

.03, 

•21, 

.07, 

.07, 

.07, 

.49) 

(. 

.01, 

.01, 

.03, 

.05, 

.09,, 

.09, 

•27, 

.45) 

( 

.02, 

.02, 

.04, 

.12, 

.08, 

.08, 

.16, 

.48) 

(. 

.01, 

.02, 

.02, 

.05, 

.09, 

.18, 

.18, 

.45) 

( 

.05, 

.05, 

.05, 

•  35, 

.05, 

.05, 

.05, 

.35) 

(. 

.02, 

.02, 

.08, 

.08, 

.08, 

.08, 

.32, 

.32) 

(. 

.02, 

.02, 

.02, 

.04, 

.18, 

.18, 

.18, 

.36) 

(. 

.04, 

.04, 

.08, 

•24, 

.06, 

.06, 

.12, 

.36) 

( 

.03, 

.03, 

.09, 

•  15, 

.07, 

.07, 

.21, 

.35) 

(. 

.02, 

.04, 

.06, 

.08, 

.08, 

.16, 

•  24, 

.32) 

(. 

.03, 

.06, 

.06, 

•15, 

.07, 

.14, 

.14, 

.35) 

(. 

.05, 

.05, 

•15, 

.25, 

.05, 

.05, 

.15, 

.25) 

(. 

.04, 

.04, 

.16, 

.16, 

.06, 

.06, 

•24, 

.24) 

(' 

.04, 

.04, 

.06, 

.06, 

.16, 

.16, 

•24, 

.24) 

(. 

.06, 

.06, 

.06, 

.12, 

.14, 

.14, 

.14, 

.28) 

( 

.04, 

.08, 

.12, 

.16, 

.06, 

•12, 

.18, 

.24) 

NUMBER  OF  EXPECTED 


MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

0056 

88 

6  6  2  2 

2  2  0 

0  0  0 

0019 

58 

6  6  6  6 

6  0  0 

0  0  0 

0013 

74 

6  6  2  2 

2  0  0 

0  0  0 

0000 

70 

8  8  0  0 

0  0  0 

0  0  0 

0377 

> 

96 

7 

7 

4 

4 

3 

3 

3 

3 

3 

3 

0221 

> 

96 

6 

6 

6 

6 

3 

3 

3 

3 

0 

0 

0213 

> 

96 

6 

6 

4 

4 

4 

4 

3 

3 

2 

2 

0201 

> 

96 

5 

5 

5 

3 

3 

3 

3 

2 

2 

0 

0192 

> 

96 

5 

5 

4 

4 

4 

4 

3 

3 

3 

1 

0169 

> 

96 

6 

6 

6 

6 

6 

6 

0 

0 

0 

0 

0133 

> 

96 

6 

6 

6 

2 

2 

2 

2 

2 

2 

0 

0131 

> 

96 

4 

4 

4 

4 

4 

4 

4 

3 

3 

0 

0117 

> 

96 

5 

5 

5 

4 

4 

2 

2 

0 

0 

0 

0105 

> 

96 

5 

5 

4 

4 

2 

2 

2 

2 

0 

0 

0099 

> 

96 

5 

5 

5 

3 

3 

2 

2 

1 

1 

0 

0090 

> 

96 

4 

4 

4 

4 

3 

1 

1 

1 

0 

0 

0069 

> 

96 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0065 

> 

96 

4 

4 

4 

4 

4 

2 

2 

0 

0 

0 

0065 

> 

96 

4 

4 

4 

4 

4 

2 

2 

0 

0 

0 

0047 

> 

96 

3 

3 

3 

3 

3 

0 

0 

0 

0 

0 

0039 

> 

96 

3 

3 

3 

2 

2 

1 

1 

0 

0 

0 

'i  AmLiSmL "V  m 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  3X3 
STATISTIC :KULLBACK 
ALPHA:  .10 

NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.04,. 06,. 10,. 06,. 09,. 15,. 10,. 15,. 25) 

.0037 

90 

6  4  3  3 

3  1 

1 

0  0  0 

(.04,. 08,. 08,. 08,. 16,. 16,. 08,. 16,. 16) 

.0021 

80 

5  5  5  5 

1  1 

1 

0  0  0 

(.06,. 06,. 08,. 09,. 09,. 12,. 15,. 15,. 20) 

.0020 

80 

6  5  5  3 

2  2 

0 

0  0  0 

(.09,. 09,. 12,. 09,. 09,. 12,. 12,. 12,. 16) 

.0005 

80 

8  4  4  0 

0  0 

0 

0  0  0 

(.11,. 11,. 11,. 11,. 11, . 11,. 11,. 11,. 11) 

.0000 

63 

9  9  9  0 

0  0 

0 

0  0  0 

(.01, .01,. 08 ,.01, .01,. 08 , . 08 , . 08 , . 64) 

.0361 

>108 

8 

8 

4 

4 

4 

4 

4. 

.4 

4 

0 

(.01,. 03,. 06,. 01,. 03,. 06,. 08,. 24,. 48) 

.0214 

>108 

7 

7 

6 

6 

4 

4 

4 

2 

2 

0 

(.01,. 02,. 07,. 02,. 04,. 14,. 07,. 14,. 49) 

.0201 

>108 

6 

6 

6 

4 

4 

4 

3 

3 

1 

0 

(.02,. 02,. 06,. 02,. 02,. 06,. 16,. 16,. 48) 

.0199 

>108 

6 

6 

6 

6 

4 

4 

4 

4 

0 

0 

(.02,. 04,. 04,. 02,. 04,. 04,. 16,. 32,. 32) 

.0141 

>108 

6 

6 

6 

6 

6 

6 

2 

2 

0 

0 

(.01,. 04,. 05,. 02,. 08,. 10,. 07,. 28,. 35) 

.0129 

>108 

6 

6 

5 

4 

4 

3 

2 

2 

1 

0 

(.01,. 03,. 06,. 03,. 09,. 18,. 06,. 18,. 36) 

.0112 

>108 

6 

5 

5 

5 

3 

3 

3 

1 

1 

0 

(.02,. 03,. 05,. 04,. 06,. 10,. 14,. 21,. 35) 

.0105 

>108 

5 

5 

5 

5 

4 

3 

2 

1 

0 

0 

(.02,. 02,. 06,. 06,. 06,. 18,. 12,. 12,. 36) 

.0101 

>108 

5 

5 

5 

5 

2 

2 

2 

2 

0 

0 

(.04,. 04,. 12,. 04,. 04,. 12,. 12,. 12,. 36) 

.0092 

>108 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

(.03,. 03,. 04,. 06,. 06,. 08,. 21,. 21,. 28) 

.0081 

>108 

6 

6 

5 

5 

3 

3 

2 

0 

0 

0 

(.01,. 04,. 05,. 04,. 16,. 20,. 05,. 20,. 25) 

.0073 

>108 

5 

5 

5 

5 

5 

3 

1 

1 

1 

0 

(.02,. 04,. 04,. 06,. 12,. 12,. 12,. 24,. 24) 

.0061 

>108 

4 

4 

4 

4 

3 

3 

1 

1 

0 

0 

(.02,. 03,. 05,. 08,. 12,. 20,. 10,. 15,. 25) 

.0054 

>108 

4 

4 

3 

3 

3 

2 

2 

1 

0 

0 

(.04,. 08,. 08,. 04,. 08,. 08,. 12,. 24,. 24) 

.0053 

>108 

6 

6 

2 

2 

2 

2 

0 

0 

0 

0 

(.03,. 03,. 04,. 12,. 12,. 16,. 15,. 15,. 20) 

.0035 

>108 

3 

3 

3 

3 

3 

3 

2 

0 

0 

0 

TABLE:  2X5 
STATISTIC  J  KULLBACK 
ALPHA:  .10 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


P  VECTOR 


(. 

.04, 

.08, 

.08, 

.08, 

.12, 

.06, 

•12, 

.12, 

•12, 

.18) 

(. 

.10, 

.10, 

.10, 

.10, 

.10, 

.10, 

.10, 

.10, 

.10, 

.10) 

(. 

.01, 

.01, 

.01, 

.01, 

.06, 

.09, 

.09, 

.09, 

.09, 

.54) 

(. 

.03, 

.03, 

.03, 

.03, 

*  18 , 

.07, 

.07, 

.07, 

.07, 

.42) 

(. 

.01, 

.01, 

.01, 

.03, 

.04, 

.09, 

.09, 

.09, 

.27, 

.36) 

(. 

.02, 

.02, 

.02, 

.04, 

.10, 

.08, 

.08, 

.08, 

.16, 

.40) 

(. 

.05, 

.05, 

.05, 

.05, 

.30, 

.05, 

.05, 

.05, 

.05, 

.30) 

(. 

.01, 

.01, 

.02, 

.03, 

.03, 

.09, 

.09, 

.18, 

.27, 

.27) 

(. 

.02, 

.02, 

.04, 

.04, 

.08, 

.08, 

.08, 

.16, 

.16, 

.32) 

(' 

.04, 

.04, 

.04, 

.08, 

.20, 

.06, 

.06, 

.06, 

.12, 

.30) 

(. 

.03, 

.03, 

.03, 

.09, 

.12, 

.07, 

.07, 

.07, 

.21, 

.28) 

(. 

.02, 

.04, 

.04, 

.04, 

.06, 

.08, 

.16, 

.16, 

.16, 

.24) 

( 

.05, 

.05, 

.05, 

.15, 

.20, 

.05, 

.05, 

.05, 

.15, 

.20) 

(. 

.03, 

.03, 

.06, 

.09, 

.09, 

.07, 

.07, 

.14, 

.21, 

.21) 

(' 

.04, 

.04, 

.08, 

.08, 

.16, 

.06, 

.06, 

•  12, 

.12, 

.24) 

(. 

.05, 

.05, 

.10, 

•15, 

•15, 

.05, 

.05, 

.10, 

.15, 

.15) 

NUMBER  OF  EXPECTED 


MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

0014 

95 

5  5  5  2 

2  1  1 

0  0  0 

0000 

79 

101010  0 

0  0  0 

0  0  0 

0228 

>100 

9 

5 

5 

5 

4 

4 

4 

4 

4 

0 

0132 

>100 

8 

8 

8 

8 

4 

4 

4 

0 

0 

0 

0130 

>100 

8 

•5 

5 

5 

5 

5 

4 

3 

3 

0 

0118 

>100 

7 

7 

4 

4 

4 

4 

3 

3 

0 

0 

0100 

>100 

8 

8 

8 

8 

8 

0 

0 

0 

0 

0 

0097 

•>100 

7 

5 

5 

5 

5 

5 

5 

3 

2 

0 

0077 

>100 

7 

7 

4 

4 

4 

4 

2 

2 

0 

0 

0066 

>100 

7 

7 

6 

6 

3 

3 

0 

0 

0 

0 

0062 

>100 

7 

6 

6 

6 

3 

3 

3 

0 

0 

0 

0050 

>100 

6 

6 

5 

5 

4 

4 

1 

1 

0 

0 

0040 

>100 

6 

6 

6 

6 

6 

0 

0 

0 

0 

0 

0039 

>100 

7 

5 

5 

5 

2 

2 

2 

0 

0 

0 

0035 

>100 

6 

6 

4 

4 

2 

2 

0 

0 

0 

0 

0020 

>100 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0 

TABLE:  2X2 
STATISTIC : KULLBACK 
ALPHA:  .05 

P  VECTOR 

(.04, .16, .16,. 64) 
(.04, .06, .36, .54) 
(.05, .05,. 45, .45) 
(.06, .14, .24, .56) 
(.08, .12, .32, .48) 
(.10, .10, .40, .40) 
(.09, .21, .21, .49) 
(.12, .18, .28, .42) 
(.15, .15, .35, .35) 
(.16, .24, .24, .36) 
(.20,. 20,. 30,. 30) 
(.25, .25, .25, .25) 


JRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0531 

96 

1 

1 

1 

1 

1 

1 

1 

0 

0 

0 

0441 

96 

2 

2 

2 

2 

2 

1 

1 

0 

0 

0 

0400 

96 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

0361 

75 

1 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0259 

54 

2 

2 

2 

2 

1 

1 

0 

0 

0 

0 

0225 

48 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

0216 

46 

3 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0129 

38 

2 

2 

2 

2 

1 

1 

0 

0 

0 

0 

0100 

36 

2 

2 

2 

2 

2 

0 

0 

0 

0 

0 

0051 

36 

3 

3 

1 

1 

1 

0 

0 

0 

0 

0 

0025 

37 

2 

2 

2 

0 

0 

0 

0 

0 

0 

0 

0000 

37 

4 

0 

0 

0 

0 

0 

0 

0 

0 

0 

(.01, .09, .09, .81)  .1056  >  96 
(.02, .08, .18, .72)  .0769  >  9b 
(.03, .07, .27, .63)  .0564  >  96 


3  3  11111111 
2  2  2  1  1  1  1  1  1  0 
2  2  2  2  1  1  1  1  0  0 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X3 
STATIST IC : KULLB ACK 
ALPHA:  .05 

NUMBER  OF  EXPECTED 

P  VECTOR  MU2  N(MIN)  VALUES  LESS  THAN 

10  987654321 


(.06,. 06,. 08,. 24,. 24,. 32) 

.0108 

96 

3 

3 

3 

2 

2 

0 

0 

0 

0 

0 

(.05, .15, .30, .05, .15, .30) 

.0106 

90 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.10, .10, .30, .10, .10, .30) 

.0089 

62 

4 

4 

4 

4 

0 

0 

0 

0 

0 

0 

(.04, .16, .20, .06, .24, .30) 

.0086 

69 

2 

2 

2 

2 

2 

2 

1 

1 

0 

0 

(.06, .12, .12, .14, .28, .28) 

.0070 

80 

3 

1 

1 

1 

1 

1 

0 

0 

0 

0 

(.08, .12, .20, .12, .18, .30) 

.0052 

48 

5 

4 

3 

3 

3 

1 

1 

0 

0 

0 

(.10, .20, .20, .10, .20, .20) 

.0022 

55 

2 

2 

2 

2 

2 

0 

0 

0 

0 

0 

(.12, .12, .16, .18, .18, .24) 

.0017 

45 

5 

5 

3 

2 

2 

0 

0 

0 

0 

0 

(.17, .17, .17, .17, .17, .17) 

.0000 

38 

6 

6 

6 

6 

0 

0 

0 

0 

0 

0 

(.01, .01, .08, .09, .09, .72) 

.0624 

> 

96 

5 

5 

3 

2 

2 

2 

2 

2 

2 

2 

(.03, .03, .24, .07, .07, .56) 

.0360 

> 

96 

4 

4 

4 

4 

2 

2 

2 

2 

0 

0 

(.01, .03, .06, .09, .27, .54) 

.0351 

> 

96 

4 

4 

3 

3 

3 

2 

2 

2 

1 

1 

(.02, .04, .14, .08, .16, .56) 

.0334 

> 

96 

3 

3 

3 

2 

2 

2 

2 

1 

1 

0 

(.02, .02, .06, .18, .18, .54) 

.0324 

> 

96 

3 

3 

3 

3 

3 

2 

2 

2 

2 

0 

(.05, .05, .40, .05, .05, .40) 

.0272 

> 

96 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

(.02, .04, .04, .18, .36, .36) 

.0214 

> 

96 

3 

3 

3 

3 

3 

3 

3 

1 

1 

0 

(.02, .08, .10, .08, .32, .40) 

.0198 

> 

96 

4 

3 

3 

1 

1 

1 

1 

1 

1 

0 

(.04, .08, .28, .06, .12, .42) 

.0190 

> 

96 

3 

3 

3 

2 

2 

1 

1 

0 

0 

0 

(.03, .09, .18, .07, .21, .42) 

.0167 

> 

96 

3 

3 

2 

2 

1 

1 

1 

1 

0 

0 

(.04, .06, .10, .16, .24, .40) 

.0153 

> 

96 

3 

2 

2 

2 

2 

1 

1 

0 

0 

0 

(.06, .06, .18, .14, .14, .42) 

.0148 

> 

96 

2 

2 

2 

2 

2 

0 

0 

0 

0 

0 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X4 

STATISTIC : KULLBACK 

ALPHA:  .05 

NUMBER 

OF 

EXPECTED 

P  VECTOR 

MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

(.05,. 10,. 10,. 25,. 05,. 10,. 10,. 25) 

.0056 

96 

6 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.06,. 06,. 06,. 12,. 14,. 14,. 14,. 28) 

.0047 

96 

3 

3 

3 

3 

3 

0 

0 

0 

0 

0 

( . 10, . 10, . 10, . 20, . 10, . 10, . 10, .20) 

.0019 

79 

6 

6 

6 

0 

0 

0 

0 

0 

0 

0 

(.08,. 08,. 12,. 12,. 12,. 12,. 18,. 18) 

.0013 

82 

6 

2 

2 

2 

0 

0 

0 

0 

0 

0 

( . 13, . 13, . 13, . 13, . 13 , . 13, . 13, . 13) 

.0000 

80 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

(.01,. 01,. 01,. 07,. 09,. OS,. 09,. 63) 

.0377 

> 

96 

7 

7 

4 

4 

3 

3 

3 

3 

3 

3 

(.03,. 03,. 03,. 21,. 07,. 07,. 07,. 49) 

.0221 

> 

96 

6 

6 

6 

6 

3 

3 

3 

3 

0 

0 

(.01,. 01,. 03,. 05,. 09,. 09,. 27,. 45) 

.0213 

> 

96 

6 

6 

4 

4 

4 

4 

3 

3 

2 

2 

(.01,. 02,. 02,. 05,. 09,. 18,. 18,. 45) 

.0192 

> 

96 

5 

5 

4 

4 

4 

4 

3 

3 

3 

1 

( .02 , . 02 , .04 , . 12, . 08, . 08 ,.16, .48) 

.0201 

> 

96 

5 

5 

5 

3 

3 

3 

3 

2 

2 

0 

(.05,. 05,. 05,. 35,. 05,. 05,. 05,. 35) 

.0169 

> 

96 

6 

6 

6 

6 

6 

6 

0 

0 

0 

0 

(.02,. 02,. 08,. 08,. 08,. 08,. 32,. 32) 

.0133 

> 

96 

6 

6 

6 

2 

2 

2 

2 

2 

2 

0 

(.02,. 02,. 02,. 04,. 18,. 18,. 18,. 36) 

.0131 

> 

96 

4 

4 

4 

4 

4 

4 

4 

3 

3 

0 

(.04,. 04,. 08,. 24,. 06,. 06,. 12,. 36) 

.0117 

> 

96 

5 

5 

5 

4 

4 

2 

2 

0 

0 

0 

( . 03, . 03, .09, .15, .07,. 07 , . 21 , . 35 ) 

.0105 

> 

96 

5 

5 

4 

4 

2 

2 

2 

2 

0 

0 

(.02,. 04,. 06,. 08,. 08,. 16,. 24,. 32) 

.0099 

> 

96 

5 

5 

5 

3 

3 

2 

2 

1 

1 

0 

(.03,. 06,. 06,. 15,. 07,. 14,. 14,. 35) 

.0090 

> 

96 

4 

4 

4 

4 

3 

1 

1 

1 

0 

0 

(.05,. 05,. 15,. 25,. 05,. 05,. 15,. 25) 

.0069 

> 

96 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

(.04,. 04,. 16,. 16,. 06,. 06,. 24,. 24) 

.0065 

> 

96 

4 

4 

4 

4 

4 

2 

2 

0 

0 

0 

(.04,. 04,. 06,. 06,. 16,. 16,. 24,. 24) 

.0065 

> 

96 

4 

4 

4 

4 

4 

2 

2 

0 

0 

0 

(.04,. 08,. 12,. 16,. 06,. 12,. IS,. 24) 

.0039 

> 

96 

3 

3 

3 

2 

2 

1 

1 

0 

0 

0 

TABLE:  3X3 
STATISTIC : KULLBACK 
ALPHA:  .05 


(.01, 

(.01, 

(.01, 

(.02, 

(.02, 

(.01, 

(.01, 

(.02, 

(.02, 

(.04, 

(.03, 

(.01, 

(.02, 

(.02, 

(.04, 

(.03, 


P  VECTOR 

MU  2 

H(hIN) 

,.10,. 06,. 09,. 15,. 10,. 15,. 25) 

.0037 

72 

, . 08, . 08, .16, .16,. 08 , . 16, . 16) 

.0021 

80 

, . 08, . 09 , . 09 ,.12, .15, .15,. 20) 

.0020 

74 

,.12,. 09,. 09,. 12,. 12,. 12,. 16) 

.0005 

64 

,.11, .11, .11, .11, .11, .11, .11) 

.0000 

51 

, . 08, .01, .01, .08 , .08, . 08 , . 64) 

.0361 

>108 

, .06, .01 , . 03, .06, .08 , . 24 , .48) 

.0214 

>108 

, .07, .02, .04, . 14, .07, . 14, .49) 

.0201 

>108 

, .06, .02, .02, .06, .16, .16, .48) 

.0199 

>108 

,.04,. 02,. 04,. 04,. 16,. 32,. 32) 

.0141 

>108 

,.05,. 02,. 08,. 10,. 07,. 28,. 35) 

.0129 

>108 

, . 06 , . 03, . 09 , . 18 , . 06 , . 18, .36) 

.0112 

>108 

,.05,. 04,. 06,. 10,. 14,. 21,. 35) 

.0105 

>108 

, . 06 , . 06 , . 06 , . 18 ,.12, .12,. 36 ) 

.0101 

>108 

,.12,. 04,. 04,. 12,. 12,. 12,. 36) 

.0092 

>108 

,.04,. 06,. 06,. 08,. 21,. 21,. 28) 

.0081 

>108 

,.05,. 04,. 16,. 20,. 05,. 20,. 25) 

.0073 

>106 

,.04,. 06,. 12,. 12,. 12,. 24,. 24) 

.0061 

>108 

,.05,. 08,. 12,. 20,. 10,. 15,. 25) 

.0054 

>108 

,.08,. 04,. 08,. 08,. 12,. 24,. 24) 

.0053 

>108 

i,. 04,. 12,. 12,. 16,. 15,. 15,. 20) 

.0035 

>108 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 
VALUES  LESS  THAN 
10  987654321 


5555111000 

6655320000 

8884400000 

9999900000 


8  8  4 
7  7  6 
6  6  6 
6  6  6 
6  6  6 
6  6  5 
6  5  5 
5  5  5 

5  5  5 

4  4  4 

6  6  5 

5  5  5 
4  4  4 
4  4  3 

6  6  2 
3  3  3 


4  4  4 
6  4  4 
4  4  4 
6  4  4 
6  6  6 

4  4  3 

5  3  3 
5  4  3 
5  2  2 

4  4  4 

5  3  3 
5  5  3 
4  3  3 
3  3  2 
2  2  2 
3  3  3 


4  4  4  0 
4  2  2  0 

3  3  10 

4  4  0  0 
2  2  0  0 
2  2  10 
3  110 
2  10  0 
2  2  0  0 
0  0  0  0 
2  0  0  0 
1110 
110  0 
2  10  0 
0  0  0  0 
2  0  0  0 


-***:,>■  ■  - 


* 


TICAL  EXPECTED  VALUE  DISTRIBUTION 


TABLE:  2X5 
STATISTIC : KULLBACK 
ALPHA:  .05 


P  VECTOR 


N(MIN) 


( 

.04, 

.08, 

.08, 

.08, 

.12, 

.06, 

.12, 

.12, 

.12, 

.18) 

(. 

.10, 

.10, 

.  10, 

.10, 

.10, 

.10, 

.10, 

.10, 

.10, 

.10) 

( 

.01, 

.01, 

.01, 

.01, 

.06, 

.09, 

.09, 

.09, 

.09, 

.54) 

< 

.03, 

.03, 

.03, 

.03, 

.18, 

.07, 

.07, 

.07, 

.07, 

.42) 

( 

.01, 

.01, 

.01, 

.03, 

.04, 

.09, 

.09, 

.09, 

.27, 

.36) 

(. 

.02, 

.02, 

.02, 

.04, 

.  10 , 

.08, 

.08, 

.08, 

.16, 

.40) 

( 

.05, 

.05, 

.05, 

.05, 

.30, 

.05, 

.05, 

.05, 

.05, 

.30) 

( 

.01, 

.01, 

.02, 

.03, 

.03, 

.09, 

.09, 

.18, 

•  27, 

.27) 

(. 

.02, 

.02, 

.04, 

.04, 

.08, 

.08, 

.08, 

.16, 

.16, 

.32) 

< 

.04, 

.04, 

.04, 

.08, 

.20, 

.06, 

.06, 

.06, 

.12, 

.30) 

(. 

.03, 

.03, 

.03, 

.09, 

.12, 

.07, 

.07, 

.07, 

•21, 

.28) 

( 

.02, 

.04, 

.04, 

.04, 

.06, 

.08, 

.16, 

.16, 

.16, 

.24) 

(. 

.05, 

.05, 

.05, 

.15, 

.20, 

.05, 

.05, 

.05, 

.15, 

.20) 

(. 

.03, 

.03, 

.06, 

.09, 

.09, 

.07, 

.07, 

.14, 

.21, 

.21) 

c 

.04, 

.04, 

.08, 

.08, 

.16, 

.06, 

.06, 

.12, 

.12, 

.24) 

< 

.05, 

.05, 

.10, 

•  15, 

.15, 

.05, 

.05, 

.10, 

.15, 

.15) 

.0014 

.0000 


.0228 

.0132 

.0130 

.0118 

.0100 

.0097 

.0077 

.0066 

.0062 

.0050 

.0040 

.0039 

.0035 

.0020 


NUMBER  OF  EXPECTED 
VALUES  LESS  THAN 
10  987654321 
5552211000 
1010  00000000 


9  5  5 
8  8  8 
8  5  5 

7  7  4 

8  8  8 
7  5  5 
7  7  4 
7  7  6 
7  6  6 

6  6  5 
6  6  6 

7  5  5 
6  6  4 
4  4  4 


5  4  4 
8  4  4 
5  5  5 

4  4  4 
8  8  0 

5  5  5 

4  4  4 

6  3  3 
6  3  3 

5  4  4 

6  6  0 
5  2  2 
4  2  2 
4  4  0 


4  4  4  0 
4  0  0  0 

4  3  3  0 
3  3  0  0 
0  0  0  0 

5  3  2  0 
2  2  0  0 
0  0  0  0 
3  0  0  0 
110  0 
0  0  0  0 
2  0  0  0 
0  0  0  0 
0  0  0  0 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


TABLE:  2X2 
STATISTIC : KULLBACK 
ALPHA:  .01 


NUMBER 


EXPECTED 


P  VECTOR 

MU  2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.04,. 16, .16, .64) 

.0531 

84 

1 

1 

1 

1 

1 

1 

1 

0 

0 

0 

(.06, .14, .24, .56) 

.0361 

70 

2 

1 

1 

1 

1 

1 

0 

0 

0 

0 

(.08, .12, .32, .48) 

.0259 

52 

2 

2 

2 

2 

1 

1 

0 

0 

0 

0 

(.10, .10,. 40, .40) 

.0225 

48 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.09, .21, .21, .49) 

.0216 

40 

3 

3 

1 

1 

1 

1 

1 

0 

0 

0 

(.12, .18, .28, .42) 

.0129 

30 

3 

3 

2 

2 

2 

1 

1 

0 

0 

0 

(.15, .15, .35, .35) 

.0100 

30 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.16, .24, .24, .36) 

.0051 

22 

4 

4 

4 

3 

3 

1 

1 

0 

0 

0 

(.20,. 20, .30, .30) 

.0025 

21 

4 

4 

4 

4 

2 

2 

0 

0 

0 

0 

(.25,. 25,. 25,. 25) 

.0000 

20 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0 

(.01, .09, .09, .81) 

.1056 

> 

96 

3 

3 

1 

1 

1 

1 

1 

1 

1 

1 

(.02, .08, .18, .72) 

.0769 

> 

96 

2 

2 

2 

1 

1 

1 

1 

1 

1 

0 

(.03, .07, .27, .63) 

.0564 

> 

96 

2 

2 

2 

2 

1 

1 

1 

1 

0 

0 

(.04, .06, .36, .54) 

.0441 

> 

96 

2 

2 

2 

2 

2 

1 

1 

0 

0 

0 

(.05, .05, .45, .45) 

.0400 

> 

96 

2 

2 

2 

2 

2 

2 

0 

0 

b 

0 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X3 
STATISTIC : KULLBACK 
ALPHA:  .01 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.04, .08, .28, .06, .12, .42) 

.0190 

84 

3 

3 

3 

3 

2 

1 

1 

0 

0 

0 

(.03, .09, .18, .07, .21, .42) 

.0167 

45 

5 

4 

3 

3 

3 

3 

2 

1 

1 

0 

(.04,. 06, .10, .16, .24, .40) 

.0153 

36 

5 

5 

4 

4 

4 

3 

3 

2 

1 

0 

(.06, .06, .18, .14, .14, .42) 

.0148 

50 

5 

4 

4 

4 

2 

2 

2 

0 

0 

0 

(.06, .06, .08, .24, .24, .32) 

.0108 

89 

3 

3 

3 

2 

2 

0 

0 

0 

0 

0 

(.05, .15, .30, .05, .15, .30) 

.0106 

82 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

(.10, .10, .30, .10, .10, .30) 

.0089 

67 

4 

4 

4 

4 

0 

0 

0 

0 

0 

0 

(.04, .16, .20, .06, .24, .30) 

.0086 

50 

3 

3 

2 

2 

2 

2 

2 

1 

0 

0 

(.06, .12, .12, .14, .28, .28) 

.0070 

75 

3 

1 

1 

1 

1 

1 

0 

0 

0 

0 

(.08, .12, .20, .12, .18, .30) 

.0052 

36 

5 

5 

5 

4 

3 

3 

1 

1 

0 

0 

(.10,. 20, .20, .10, .20, .20) 

.0022 

44 

6 

6 

2 

2 

2 

2 

0 

0 

0 

0 

(.12, .12, .16, .18, .18, .24) 

.0017 

35 

6 

6 

5 

5 

3 

2 

0 

0 

0 

0 

(.17, .17, .17, .17, .17, .17) 

.0000 

36 

6 

6 

6 

6 

0 

0 

0 

0 

0 

0 

( .01, .01, .08, .09, .09, .72) 

.0624 

> 

96 

5 

5 

3 

2 

2 

2 

2 

2 

2 

2 

(.03, .03, .24, .07, .07, .56) 

.0360 

> 

96 

4 

4 

4 

4 

2 

2 

2 

2 

0 

0 

(.01, .03, .06, .09, .27, .54) 

.0351 

> 

96 

4 

4 

3 

3 

3 

2 

2 

2 

1 

1 

(.02, .04, .14, .08, .16, .56) 

.0334 

> 

96 

3 

3 

3 

2 

2 

2 

2 

1 

1 

0 

(.02, .02, .06, .18, .18, .54) 

.0324 

> 

96 

3 

3 

3 

3 

3 

2 

2 

2 

2 

0 

(.05, .05, .40, .05, .05, .40) 

.0272 

> 

96 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

C • 02, • 04 , • 04 , • 18, • 36 , • 36) 

.0214 

> 

96 

3 

3 

3 

3 

3 

3 

3 

1 

1 

0 

( *02, *08, *10, *08, *32, *40) 

.0198 

> 

96 

4 

3 

3 

1 

1 

1 

1 

1 

1 

0 

TABLE:  2X4 
STATISTIC : KULLBACK 
ALPHA:  .01 


(.04, 

(.03, 

(.03, 

(.05, 

(.05, 

(.06, 

(.04, 

(.10, 

(.08, 

(.13, 


( *01, . 
(.03,. 
(.01,. 
( *02, . 
(.01,. 
( .05, . 
(.02,. 
(.02,. 
(.02,. 
( .04, . 
( *04,  • 


P  VECTOR 

MU2 

N(MI1 

! , . 24, . 06 , . 06 , . 12, . 36) 

.0117 

78 

>,.15, .07, .07, .21, .35) 

.0105 

50 

i,. 15, .07, .14, .14, .35) 

.0090 

40 

> , . 25 ,. 05,. 05, .15,. 25 ) 

.0069 

85 

1,. 25, .05, .10, .10, .25) 

.0056 

82 

i,. 12, .14, .14, .14, .28) 

.0047 

82 

:,. 16, .06, .12, .18, .24) 

.0039 

72 

>,. 20,  .10,  .10, .10,. 20) 

.0019 

70 

1,.  12,  .12,  .12,  .18,.  18) 

.0013 

80 

,.13, .13, .13, .13, .13) 

.0000 

70 

,.07, .09, .09, .09, .63) 

.0377 

> 

96 

,.21, .07, .07, .07, .49) 

.0221 

> 

96 

,.05, .09, .09, .27, .45) 

.0213 

> 

96 

,.12, .08, .08, .16, .48) 

.0201 

> 

96 

,.05, .09, .18, .18,. 45 ) 

.0192 

> 

96 

,.35, .05, .05, .05, .35) 

.0169 

> 

96 

, . 08 , . 08 , . 08 , .32, .32) 

.0133 

> 

96 

,.04, .18, .18, .18, .36) 

.0131 

> 

96 

,.08, .08, .16, .24, .32) 

.0099 

> 

96 

,.06, .16, .16, .24, .24) 

.0065 

> 

96 

,.16, .06, .06, .24, .24) 

.0065 

> 

96 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 
VALUES  LESS  THAN 
10  987654321 
6  5  5  5  4  4  2  0  0  0 
6665554220 
7777644410 
4444440000 
6622220000 
4333330000 
5533321100 
'6  660000000 
6222000000 
8800000000 


7744333333 

6666333300 

6644443322 

5553333220 

5544443331 

6666660000 

6662222220 

4444444330 

5553322110 

4444422000 

4444422000 


,«,r 


V 


TABLE:  3X3 

STATISTIC : KULLBACK 

ALPHA:  .01 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

NUMBER  OF  EXPECTED 

P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.01,. 04,. 05,. 04,. 16,. 20,. 05,. 20,. 25) 

.0073 

90 

5 

5 

5 

5 

5 

5 

3 

1 

1 

1 

(.02,. 04,. 04,. 06,. 12,. 12,. 12,. 24,. 24) 

.0061 

96 

4 

4 

4 

4 

4 

3 

3 

1 

1 

0 

(.04,. 06,. 10,. 06,. 09,. 15,. 10,. 15,. 25) 

.0037 

74 

6 

6 

6 

4 

3 

3 

1 

1 

0 

0 

(.03,. 03,. 04,. 12,. 12,. 16,. 15,. 15,. 20) 

.0035 

78 

5 

3 

3 

3 

3 

3 

3 

2 

0 

0 

(.04,. 08,. 08,. 08,. 16,. 16,. 08,. 16,. 16) 

.0021 

72 

5 

5 

5 

5 

5 

1 

1 

1 

0 

0 

(.06,. 06,. 08,. 09,. 09,. 12,. 15,. 15,. 20) 

.0020 

72 

6 

6 

5 

5 

3 

2 

0 

0 

0 

0 

(.09,. 09,. 12,. 09,. 09,. 12,. 12,. 12,. 16) 

.0005 

67 

8 

8 

4 

4 

0 

0 

0 

0 

0 

0 

(.11,. 11,. 11,. 11,. 11,. 11,. 11,. 11,. 11) 

.0000 

51 

9 

9 

9 

9 

9 

0 

0 

0 

0 

0 

( .01, .01, .08, .01, .01, .08, . 08 , .08, .64) 

.0361 

>108 

8 

8 

4 

4 

4 

4 

4 

4 

4 

0 

(.01, .03,. 06 ,.01, .03,. 06 , . 08 , . 24 , . 48 ) 

.0214 

>108 

7 

7 

6 

6 

4 

4 

4 

2 

2 

0 

(.01,. 02,. 07,. 02,. 04,. 14,. 07,. 14,. 49) 

.0201 

>108 

6 

6 

6 

4 

4 

4 

3 

3 

1 

0 

(.02,. 02,. 06,. 02,. 02,. 06,. 16,. 16,. 48) 

.0199 

>108 

6 

6 

6 

6 

4 

4 

4 

4 

0 

0 

( . 02, . 04, . 04, . 02 , . 04 , . 04 , . 16 , . 32 , « 32 ) 

.0141 

>108 

6 

6 

6 

6 

6 

6 

2 

2 

0 

0 

( . 01 , . 04 , . 05 , . 02 , . 08 , . 10 , . 07 , .28, .35) 

.0129 

>108 

6 

6 

5 

4 

4 

3 

2 

2 

1 

0 

(.01,. 03,. 06,. 03,. 09,. 18,. 06,. 18,. 36) 

.0112 

>108 

6 

5 

5 

5 

3 

3 

3 

1 

1 

0 

(.02.. 03,. 05,. 04,. 06,. 10,. 14,. 21,. 35) 

.0105 

>108 

5 

5 

5 

5 

4 

3 

2 

1 

0 

0 

( . 02 , . 02 , . 06 , . 06 , . 06 ,.18, .12, .12, .36) 

.0101 

>108 

5 

5 

5 

5 

2 

2 

2 

2 

0 

0 

(.04,. 04,. 12,. 04,. 04,. 12,. 12,. 12,, 36) 

.0092 

>108 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

(.03,. 03,. 04,. 06,. 06,. 08,. 21,. 21,. 28) 

.0081 

>108 

6 

6 

5 

5 

3 

3 

2 

0 

0 

0 

(.02,. 03,. 05,. 08,. 12,. 20,. 10,. 15,. 25) 

.0054 

>108 

4 

4 

3 

3 

3 

2 

2 

1 

0 

0 

(.04,. 08,. 08,. 04,. 08,. 08,. 12,. 24,. 24) 

.0053 

>108 

6 

6 

2 

2 

2 

2 

0 

0 

0 

0 

TABLE:  2X5 
STATISTIC : KULLBACK 
ALPHA:  .01 


P  VECTOR 


(.01,. 01,. 
(.03, .03,. 
( . 01 , . 01 , . 
( . 02, . 02, < 
(.05, .05,. 
(.01, .01,. 
(.02,. 02,. 
(.04,. 04,, 
( . 03, . 03, < 
(.02,. 04, 
(.03, .03, 


,.15, 

.20, 

.05, 

.  05 , . ' 

,  .08, 

.16,. 

.06, 

.06,. 

,.15, 

.15, 

.05, 

.05,. 

,  .08, 

.12, 

.06, 

.12,. 

,.io, 

.10,. 

.10, 

.10,. 

,.01, 

.06, 

.09, 

.09,. 

»  .03, 

.18, 

.07, 

•  07 , .  i 

,  *03, 

.04, 

.09, 

.09,. 

,.04, 

.10, 

.08, 

.  08 , . 

,.05, 

.30, 

.05, 

•  05 , . 

,.03, 

.03, 

.09, 

.09,. 

,.04, 

.08, 

.08, 

.  08 , . 

,.08, 

.20, 

.06, 

.06,. 

,.09, 

.12, 

.07, 

.07,. 

►,.04, 

.06, 

.08, 

.16,. 

',.09, 

.09, 

.07, 

.07,. 

.09,. 54) 
.07, .42) 
.27,. 36) 
.16, .40) 
.05,. 30) 
.27, .27) 
.16, .32) 
.12, .30) 
.21, .28) 
,.16, .24) 
,.21, .21) 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 
MU2  N(MIN)  VALUES  LESS  THAN 


.0040 

.0035 

.0020 

.0014 

.0000 


.0228 

.0132 

.0130 

.0118 

.0100 

.0097 

.0077 

.0066 

.0062 

.0050 

.0039 


>100 

>100 

>100 

>100 

>100 

>100 

>100 

>100 

>100 

>100 

>100 


10  987654321 
6  6  6  6  6  6  0  0  0  0 
6664422000 
6644440000 
5555211000 
10101010  000000 


9  5  5  5 
8  8  8  8 
8  5  5  5 

7  7  4  4 

8  8  8  8 
7  5  5  5 
7  7  4  4 
7  7  6  6 
7  6  6  6 
6  6  5  5 


4  4  4  4  4  0 

4  4  4  0  0  0 

5  5  4  3  3  0 

4  4  3  3  0  0 

8  0  0  0  0  0 

5  5  5  3  2  0 

4  4  2  2  0  0 

3  3  0  0  0  0 

3  3  3  0  0  0 

4  4  110  0 


7555222000 


-swag 


M 

..'fw 

‘V| 


i,j 

M 


TABLE:  S2X2X2 
STATISTIC : KULLBACK 
ALPHA:  .10 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


(.008, 

(.009, 

(.016, 

(.015, 

(.010, 

(.016, 

(.025, 

(.030, 

(.027, 

(.032, 

(.050, 

(.045, 


P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

, .108,. 112,. 168,. 168,. 252) 

.0040 

71 

5 

5 

5 

3 

3 

1 

1 

0 

0 

0 

,.075,. 175,. 175,. 175,. 175) 

.0025 

99 

4 

4 

4 

0 

0 

0 

0 

0 

0 

0 

, . 144, .096, . 144, . 144, .216) 

.0020 

74 

4 

4 

4 

1 

1 

1 

0 

0 

0 

0 

'»  .  100, .  ISO, .  150, .  150,.  150) 

.0006 

54 

8 

8 

4 

4 

4 

0 

0 

0 

0 

0 

, . 125, « 125, . 125, . 125 , . 125) 

.0000 

•  58 

8 

8 

8 

0 

0  0  0 

0 

0 

0 

.,.046, .072, . 108, .288, . 432) 

.0206 

>104 

5 

5 

5 

4 

4 

4 

3 

2 

2 

1 

, .049, .081 , . 189, . 189, .441) 

.0189 

>104 

5 

5 

4 

4 

4 

3 

3 

3 

1 

1 

.» .096, .064, .096,  .256, .384) 

.0144 

>104 

6 

4 

4 

4 

2 

2 

2 

2 

1 

0 

.035, .135, .135, .315, .315) 

.0141 

>104 

4 

4 

4 

4 

4 

4 

4 

2 

2 

0 

:,. 098,. 072,. 168,. 168,. 392) 

.0130 

>104 

4 

4 

4 

3 

3 

3 

1 

1 

1 

0 

>,.036, .144, .216, .216, .324) 

.0121 

>104 

4 

4 

4 

4 

4 

4 

4 

3 

1 

0 

>,  .025,  .225, .  225,  .225,  .225) 

.0100 

>104 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

i,  .070, .  120,  *  120,  .280,  .280) 

.0090 

>104 

4 

4 

4 

2 

2 

2 

2 

0 

0  0 

i, .  147 ,  .063 , .  147 , .  147 , .  343) 

.0088 

>104 

4 

4 

4 

4 

1 

1 

1 

1 

0 

0 

•, .072, .128, . 192, . 192, . 288) 

.0074 

>104 

4 

4 

4 

3 

3 

3 

1 

0 

0 

0 

',.050, .200,. 200, .200,. 200) 

.0056 

>104 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0 

i, .  105, .  105, .  105,  .245 , .  245) 

.0054 

>104 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

xABLE:  2X2X2 
STATISTIC :KULLBACK 
ALPHA:  .10 


P  VECTOR 


(.001, 

(.003, 

(.004, 

(.005, 

(.ooa, 

(.008, 

(.009, 

(.016, 

(.015, 

(.018, 

(.016, 

(.025, 

(.030, 

(.027, 

(.032, 

(.048, 


.009,. 
.  007 , . 
.016,. 
.005,. 
.032,. 
.  012, . 
.021,. 
.024,. 
.015,. 
.042,. 
.024,. 
.025,. 
.030,. 
.  063 , . 
.048,. 
.072,. 


009,. 081, 
027,. 063, 
016, .064, 
045,. 045, 
032, .128, 
032,. 048, 
021,. 049, 
064,. 096, 
035,. 035, 
042,. 098, 
024,. 036, 
025, .025, 
070,. 070, 
063,. 147, 
048,. 072, 
072,. 108, 


MU2 

N(MU 

,.200,. 200,. 200) 

.0056 

96 

,.105,. 245,. 245) 

.0054 

96 

,.175, .175, .175) 

.0025 

63 

,.144, .144, .216) 

.0020 

66 

,.150,. 150,. 150) 

.0006 

63 

,.125,. 125,. 125) 

.0000 

56 

,.081, .081, .729) 

.0533 

> 

96 

,.063,. 243,. 567) 

‘.0331 

> 

96 

,.144,. 144,. 576) 

.0318 

> 

96 

,.045,. 405,. 405) 

.0264 

> 

96 

,.128, .128, .512) 

.0237 

> 

96 

,.108,. 288,. 432) 

.0206 

> 

96 

,.189, .189, .441) 

.0189 

> 

96 

,.096, .256, .384) 

.0144 

> 

96 

,.135, .315, .315) 

.0141 

> 

96 

,.168, .168, .392) 

.0130 

> 

96 

,.216, .216, .324) 

.0121 

> 

96 

,.225, .225, .225) 

.0100 

> 

96 

,.120, .280, .280) 

.0090 

> 

96 

,.147, .147, .343) 

.0088 

> 

96 

,.192, .192, .288) 

.0074 

> 

96 

,.168, .168, .252) 

.0040 

> 

96 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 
VALUES  LESS  THAN 
10  987654321 
4  4  4  4  4  4  0  0  0  0 
2222220000 
4444440000 
7444110000 
8444000000 
8880000000 


7  7  7 
6  6  6 

5  5  5 

6  6  6 

4  4  4 

5  5  5 

5  5  5 

6  4  4 

4  4  4 

5  4  4 
4  4  4 
4  4  4 
4  4  4 
4  4  4 
4  4  4 
3  3  3 


4  4  4  4 
6  4  4  4 

5  4  4  4 

6  6  6  2 

4  4  4  4 

5  4  4  3 
4  4  4  3 
4  2  2  2 
4  4  4  4 
4  3  3  1 
4  4  4  4 
4  4  4  4 
4  2  2  2 
4  111 
4  3  3  1 
3  110 


4  4  4 
4  2  2 
3  3  1 
2  2  2 
1  1  1 
2  2  1 
3  1  1 
2  1  0 
2  2  0 
1  1  0 

3  1  0 

4  0  0 
2  0  0 
1  0  0 
0  0  0 
0  0  0 


» 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X2X3 
STATISTIC : KULLBACK 
ALPHA:  .10 

NUMBER  OF  EXPECTED 

P  VECTOR  MU2  N(MIN)  VALUES  LESS  THAN 

10  987654321 


(.083,. 083,. 083,. 083,. 083,. 083,. 083,. 083,. 083,. 083. 

(.001, .001, .008, .009, .009, .072, .009,. 009,. 072,. 081, 
( .002, .004, .014, .008, .016, .056, .018, .036, .126, .072, 
( .002, .002, .006, .018, .018, .054, .018, .018, .054, . 162, 
(.005,. 005,. 040,. 005,. 005,. 040,. 045,. 045,. 360,. 045, 
( .004, .008, .028, .016, .032, . 112, .016, .032, . 112, .064, 
( .010, .010, .080, .010, .010, .080, .040, .040, .320, .040, 
(.003,. 009,. 018,. 007,. 021,. 042,. 027,. 081,. 162,. 063, 
( .004, .006, .010, .016, .024, .040, .036, .054, .090, . 144, 
(.009,. 018, .063,. 021,. 042.. 147,. 021,. 042,. 147,. 049, 
( .006, .018, .036, .014, .042, .084, .024, .072, . 144, .056, 
(.010, .010, .030, .010, .010, .030, .090, .090, .270, .090, 
( . 004 , . 016 , . 020 , . 006 , . 024 , . 030 , . 036 , . 144 , . 160 , . 054, 
( .016, .016, .128, .024, .024, .192, .024, .024, .192, .036, 
( .008, .012, .020, .032, .048, .080, .032, .048, .080, . 128, 
(.006,. 012,. 012,. 014,. 028,. 028,. 054,. 108,. 108,. 126, 
( . 020 , . 020 , . 060 , . 020 , . 020 , . 060 , . 080, . 080 , . 240 , . 080 , 
(.008,. 032,. 040,. 012,. 048,. 060, .032,. 128,. 160,. 048, 
(.012,. 012,. 016,. 018,. 018, .024,. 108,. 108,. 144,. 162, 
(.012,. 024,. 024,. 028,. 056,. 056,. 048,. 096,. 096,. 112, 
(.020,. 040,. 140,. 020,. 040,. 140,. 030,. 060, .210,. 030, 
( .012, .036, .072, .018, .054, .108, .028, .084, . 168, .042, 
(.018, .027, .045, .042, .063, . 105, .042, .063, . 105, .098, 
( .015, .060, .075, .015, .060, .075, .035, . 140, . 175, .035, 
(.024, .024,. 032, .036, .036, .048, .096, .096, .128, . 144, 
( .032, .032, .096, .048, .048, .144, .048, .048, . 144, .072, 
( .025, .075, - 150, .025, .075, . 150, .025, .075, . 150, .025, 
(.024, .048, .048, .036, .072, .072, .056, . 1 12, . 112, .084, 
( .040, .060, . 100, .040, .060, . 100, .060, . 090, . 150, .060, 
(.045,. 045,. 060,. 045,. 045,. 060,. 105,. 105,. 140,. 105, 
( .050, . 100, . 100, .050, . 100, . 100, .050, . 100, . 100, .050, 


083,. 083) 

.0000 

96 

1212 

0 

0 

0  0 

0 

0 

0 

0 

081,. 648) 

.0300 

> 

96 

111111 

9 

7 

7 

7 

7 

7 

7 

144,. 504) 

.0181 

> 

96 

9 

9 

9 

9 

8 

7 

7 

6 

6 

3 

162,. 486) 

.0177 

> 

96 

9 

9 

9 

9 

9 

7 

7 

7 

7 

3 

045,. 360) 

.0156 

> 

96 

101010101010 

6 

4 

4 

4 

128,. 448) 

.0139 

> 

96 

8 

8 

8 

8 

7 

7 

7 

5 

4 

2 

040,. 320) 

.0118 

> 

96 

101010 

8 

8 

8 

8 

4 

4 

4 

189,. 378) 

.0113 

> 

96 

9 

9 

9 

8 

7 

7 

6 

6 

4 

3 

216,. 360) 

.0107 

> 

96 

9  9  8 

8 

8 

7 

7 

5 

4 

3 

098,. 343) 

.0082 

> 

96 

9  8  8 

8 

7 

7 

4 

4 

2 

1 

168,. 336) 

.0082 

> 

96 

9  9  8 

8 

7 

6 

5 

4 

3 

1 

090,. 270) 

.0081 

> 

96 

1010  6 

6 

6 

6 

6 

6 

4 

4 

216,. 270) 

.0080 

> 

96 

8  8  8 

8 

8 

7 

7 

6 

4 

2 

036,. 288) 

.0079 

> 

96 

8  8  8 

8 

8 

8 

8 

6 

2  0 

192,. 320) 

.0077 

> 

96 

9 

9 

9 

7 

7 

7 

5 

3 

3 

1 

252,. 252) 

.0073 

> 

96 

7 

7 

7 

7 

7 

6 

6 

6 

4 

1 

080,. 240) 

.0055 

> 

96 

101010 

6 

6 

4 

4 

4 

4 

0 

192,. 240) 

.0054 

> 

96 

8 

8 

8 

8 

8 

7 

5 

2 

2 

1 

162,. 216) 

.0051 

> 

96 

6 

6 

6 

6 

6 

6 

6 

6 

5 

0 

224,. 224) 

.0049 

> 

96 

9 

7 

7 

7 

7 

5 

4 

4 

1 

0 

.060,. 210) 

.0048 

> 

96 

8 

8 

8 

8 

8 

6 

6 

4 

2 

0 

126,. 252) 

.0046 

> 

96 

8 

8 

7 

7 

6 

5 

4 

3 

2 

0 

147, .245) 

.0037 

> 

96 

8 

7 

7 

7 

5 

5 

2 

2 

1 

0 

140,. 175) 

.0032 

> 

96 

8 

8 

8 

6 

6 

4 

4 

2 

2 

0 

.144, .192) 

.0031 

> 

96 

8 

6 

6 

6 

6 

6 

5 

2 

0 

0 

072, .216) 

.0030 

> 

96 

9 

8 

8 

8 

6 

6 

2 

0 

0 

0 

.075,. ISO) 

.0026 

> 

96 

8 

8 

8 

4 

4 

4 

4 

4 

0 

0 

168,. 168) 

.0021 

> 

96 

8 

8 

7 

7 

5 

4 

2 

1 

0 

0 

U90..150) 

.0013 

> 

96 

10 

8 

6 

6 

6 

2 

2 

0 

0 

0 

105,. 140) 

.0013 

> 

96 

6 

6 

6 

6 

6 

4 

0 

0 

0 

0 

100, .100) 

.0006 

> 

96 

12 

4 

4 

4 

4 

4 

0 

0 

0 

0 

P  VECTOR 

(. 048,. 072,. U72,. 108,. 112,. 168,. 168,. 252) 
(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 
(.064,. 096,. 096,. 144,. 096,. 144,. 144,. 216) 
( . 100, . 100, . 100, . 100, . 150, . 150, . 150, . 150) 
( . 125, . 125, . 125, . 125, . 125, . 125, . 125, . 125) 


NUMBER  OF  EXPECTED 


MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

.0040 

99 

3  3  3  1 

1  1  0 

0  0  0 

.0025 

94 

4  4  4  0 

0  0  0 

0  0  0 

.0020 

60 

7  7  4  4 

4  1  1 

0  0  0 

.0006 

54 

8  8  4  4 

4  0  0 

0  0  0 

.0000 

55 

8  8  8  8 

0  0  0 

0  0  0 

(. 

.008, 

.012, 

.032, 

.048, 

.072 

(. 

.009, 

.021, 

.021, 

.049, 

.081 

(. 

.016, 

.024, 

.064, 

.096, 

.064 

(. 

.015, 

.015, 

.035, 

.035, 

.135 

(' 

.018, 

.042, 

.042, 

.098, 

.072' 

(. 

.016, 

.024, 

.024, 

.036, 

.144 

(. 

.025, 

.025, 

.025, 

.025, 

.225 

(' 

.030, 

.030, 

.070, 

.070, 

.120 

(. 

.027, 

.063, 

.063, 

.147, 

•  U63 

(. 

.032, 

.048, 

.048, 

.072, 

.128 

(. 

.050, 

.050, 

.050, 

.050, 

.200 

(. 

.045, 

.045, 

.105, 

.105, 

.105 

.108,. 288,. 432)  .0206 
•  189, . 189, .441)  .0189 
.096,. 256,. 384)  .0144 
.135, .315, .315)  .0141 
.168,. 168,. 392)  .0130 
.216, .216, .324)  .0121 
.225, .225,. 225)  .0100 
.120,. 280,. 280)  .0090 
.147, .147, .343)  .0088 
.192, .192, .288)  .0074 
.200,. 200,. 200)  .0056 
.105,. 245,. 245)  .0054 


>104 

5 

5 

5 

4 

4 

4 

3 

2 

2 

1 

>104 

5 

5 

4 

4 

4 

3 

3 

3 

1 

1 

>104 

6 

4 

4 

4 

K 

?, 

2 

2 

1 

0 

>104 

4 

4 

4 

4 

U 

j 

4 

2 

0 

>104 

4 

4 

$ 

h 

.5 

'i 

3 

1 

< 

> 

t 

0 

>104 

4 

4 

i 

4 

4 

4 

4 

3 

1 

0 

>104 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

>104 

4 

4 

4 

2 

2 

2 

2 

0 

0 

0 

>104 

4 

4 

4 

4 

1 

1 

1 

1 

0 

0 

>104 

4 

4 

4 

3 

3 

3 

1 

0 

0 

0 

>104 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0 

>104 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

306 


TABLE:  2X2X2 
STATISTIC : KULLBACK 
ALPHA:  .01 

P  VECTOR 

( . 030 , .  030 , . 070 , . 070 , .  120 , .  120 , .  280, .  280) 
( .027,-063, .063, . 147, .063, . 147, . 147, .343) 
( . 050 , . 050 , . 050 , . 050 , . 200 , . 200 , . 200 , . 200) 
(.048,. 072,. 072,. 108,. 112,. 168,. 168,. 252) 
(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 
(.064,. 096,. 096,. 144,. 096,. 144,. 144,. 216) 
( . 100, . 100, . 100, . 100, . 150, . 150, . 150, . 150) 
( . 125, .125, . 125, . 125, . 125, . 125, . 125, . 125) 

(.001,. 009,. 009,. 081,. 009,. 081,. 081,. 729) 
(.003,. 007,. 027,. 063,. 027,. 063,. 243,. 567) 
(.004,. 016,. 016,. 064,. 036,. 144,. 144,. 576) 
( .005, .005, .045, .045, .045, .045, .405, .405) 
(.008,. 032,. 032,. 128,. 032,. 128,. 128,. 512) 
(.008,. 012,. 032,. 048,. 072,. 108,. 288,. 432) 
(.009,. 021,. 021,. 049,. 081,. 189,. 189,. 441) 
(.016,. 024,. 064,. 096,. 064,. 096,. 256,. 384) 
(.015,. 015,. 035,. 035,. 135,. 135,. 315,. 315) 
(.018,. 042,. 042,. 098,. 072,. 168,. 168,. 392) 
(.016,. 024,. 024,. 036,. 144,. 216,. 216,. 324) 
( . 025, . 025, . 025, . 025, . 225, .225,. 225, . 225) 
(.032,. 048,. 048,. 072,. 128,. 192,. 192,. 288) 
( .045, .045, . 105, . 105, . 105, . 105, .245, . 245) 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10  9 

11 

6 

5 

4 

3 

2 

1 

0090 

83 

6 

4 

4 

4 

4 

2 

2 

2 

0 

0 

0088 

56 

7 

7 

4 

4 

4 

4 

4 

1 

1 

0 

0056 

68 

4 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0040 

64 

5 

5 

5 

4 

3 

3 

1 

0 

0 

0 

0025 

61 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0020 

53 

7 

7 

7 

4 

4 

1 

1 

0 

0 

0 

0006 

45 

8 

8 

8 

8 

4 

4 

0 

0 

0 

0 

0000 

45 

8 

8 

8 

8 

8 

0 

0 

0 

0 

0 

0533 

> 

96 

7 

7 

7 

4 

4 

4 

4 

4 

4 

4 

0331 

> 

96 

6 

6 

6 

6 

4 

4 

4 

4 

2 

2 

0318 

> 

96 

5 

5 

5 

5 

4 

4 

4 

3 

3 

1 

0264 

> 

96 

6 

6 

6 

6 

6 

6 

2 

2 

2 

2 

0237 

> 

96 

4 

4 

4 

4 

4 

4 

4 

1 

1 

1 

0206 

> 

96 

5 

5 

5 

5 

4 

4 

3 

2 

2 

1 

0189 

> 

96 

5 

5 

5 

4 

4 

4 

3 

3 

1 

1 

0144 

> 

96 

6 

4 

4 

4 

2 

2 

2 

2 

1 

0 

0141 

> 

96 

4 

4 

4 

4 

4 

4 

4 

2 

2 

0 

0130 

> 

96 

5 

4 

4 

4 

3 

3 

1 

1 

1 

0 

0121 

> 

96 

4 

4 

4 

4 

4 

4 

4 

3 

1 

0 

0100 

> 

96 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

0074 

> 

96 

4 

4 

4 

4 

3 

3 

1 

0 

0 

0 

0054 

> 

96 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

TABLE:  2X2X3 
STATISTIC :  KULLBACK 
ALPHA:  .01 

P  VECTOR 

( .024, .04#, .048, .036, .072, .072, .036, . 112, . 112, .084, 
( .040, .060, . 100, .040, .060, . 100, .060, .090, . 130, . 060, 
(.043,. 043, .060,. 043,. 043,. 060,. 103,. 103,. 140,. 103, 
( . 083 , . 083 , . 083 , . 083 , . 083 , . 083 , . 083 , . 083 , . 083 , . 083 , 

(. 001,. 001,. 008,. 009,. 009,. 072,. 009,. 009,. 072,. 081, 
(.002,. 004,. 014,. 008,. 016,. 036,. 018, .036,. 126,. 072, 
( .002, .002. .006, .018, .018, .034, .018, .018, .034, . 162, 
( .003, .003, .040, .003, .003, .040, .043, .043, .360, .043, 
( .004, .008. .028, .016, .032, . 112, .016, .032, . 112, .064, 
( .010, .010, .080, .010, .010, .060, .040, .040, . 320, . 040, 
( .003, .OOt, .018, .007, .021, .042, .027, .081, . 162, .063, 
( .004, .006, .010, .016, .024, .040 , .036, .034, .090, . 144, 
( .009, .018, .063, .021, .042, . 147, .021, .042, . 147 , .049, 
( .006, .018, .036, .014, .042, .084 , .024, .072, . 144, . 036, 
( .010, .010, .030, .010, .010, .030, .090, .090, . 270, .090, 
( .004, .016, .020, .006, .024, .030, .036, . 144, . 180, .034, 
( .016, .016, . 128, .024, .024, . 192, .024, .024, . 192, .036, 
( .008, .012, .020, .032, .048, .080, .032, .048, .080, .128, 
( . cUw, .012, .012, .014, .028, .028, .034, . 108, . 108, . 126, 
( .020, .020, .060, .020, .020, .060, .080, .080, . 240, . 080, 
( .000, .032. .040, .012, .048. .060, .032, . 128, . 160, .048, 
( .012, .012, .016, .018, .018, .024, . 108, . 108, . 144, . 162, 
( .012, .024, .024, .028, .036, .036, .048, .096, .096, . 112, 
( .020, .040, . 140, .020, .040, . 140, .030, .060, . 210, .030, 
( .012, .036, .072, .018, .034, . 108, .028, .084, . 168, .042, 
(.016,. 027,. 043,. 042,. 063,. 103,. 042,. 063,. 103,. 098, 
(.013,. 060,. 073,. 013,. 060,. 073,. 033,. 140,. 173,. 033, 
( .024 , .024, . 032, .036, . 036 , . 048, . 096 , . 096 , . 128 , . 144 , 
( .032, .032, .096, .048, .048, . 144, .048, .048, . 144, .072, 
( .023, .073, .130,. 023,. 073, .130, .023, .073, . ISO, .023, 
( .030, .100, . 100, .030, . 100, . 100, .030, . 100, .100, .030, 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10  9 

8 

7 

6 

5 

4 

2. 

2 

1 

168,. 168) 

.0021 

88 

10  8  8 

7 

5 

3 

2 

1 

0 

0 

090,. 130) 

.0013 

67 

10101010 

6 

6 

2 

2 

0 

0 

103,. 140) 

.0013 

89 

10  6 

6 

6 

6 

4 

0 

0 

0 

0 

083,. 083) 

.0000 

69 

1212121212  000 

0 

0 

081,. 648) 

.0300 

> 

96 

111111 

9 

7 

7 

7 

7 

7 

7 

144,. 504) 

.0181 

> 

96 

9  9 

9 

9 

8 

7 

7 

6 

6 

3 

162,. 486) 

.0177 

> 

96 

9  9 

9 

9 

9 

7 

7 

7 

7 

3 

045,. 360) 

.0156 

> 

96 

101010101010  6 

4 

4 

4 

128,. 448) 

.0139 

> 

96 

8  8 

8 

8 

7 

7 

7 

5 

4 

2 

040,. 320) 

.0118 

> 

96 

101010  88884 

4 

4 

189,. 378) 

.0113 

> 

96 

9  9  9 

8 

7 

7 

6 

6 

4 

3 

216,. 360) 

.0107 

> 

96 

9  9  8 

8 

8 

7 

7 

5 

4 

3 

098,. 343) 

.0082 

> 

96 

9  8 

8 

8 

7 

7 

4 

4 

2 

1 

168,. 336) 

.0082 

> 

96 

9  9 

8 

8 

7 

6 

5  4 

3 

1 

090,. 270) 

.0081 

>  96 

1010  6 

6 

6 

6 

6 

6 

4 

4 

216,. 270) 

.0080 

> 

96 

8  8  8  8  8 

7 

7 

6 

4 

2 

036,. 288) 

.0079 

> 

96 

8  8 

8 

8 

8 

8 

8 

6 

2 

0 

192.. 320) 

.0077 

> 

96 

9  9 

9 

7 

7 

7 

5 

3 

3 

1 

232,. 252) 

.0073 

> 

96 

7  7 

7 

7 

7 

6 

6 

6 

4 

1 

080,. 240) 

.0055 

> 

96 

101010  6 

6 

4 

4 

4 

4 

0 

192, .240) 

.0054 

> 

96 

8  8 

8 

8  8 

7 

5 

2 

2 

1 

162, .216) 

.0051 

> 

96 

6  6 

6 

6 

6 

6 

6 

6 

5 

0 

224,. 224) 

.0049 

> 

96 

9  7 

7 

7 

7 

5 

4 

4 

1 

0 

060,. 210) 

.0048 

> 

96 

8  6 

8 

8 

8 

6 

6 

4 

2 

0 

126,. 232) 

.0046 

> 

96 

8  8 

7 

7 

6 

5 

4 

3 

2 

0 

147, .245) 

.0037 

> 

96 

8  7 

7 

7 

5 

5 

2 

2 

1 

0 

140,. 175) 

.0032 

> 

96 

8  8 

8 

6 

6 

4 

4 

2 

V 

144,. 192) 

.0031 

> 

96 

8  6 

6 

6 

6 

6 

5 

2 

0 

0 

072, .216) 

.0030 

> 

96 

9  6 

8 

8 

6 

6 

2 

0 

0 

u 

073,. 150) 

.0026 

> 

96 

8  8 

8 

4 

4 

4 

4 

4 

0 

0 

100,. 100) 

.0006 

> 

96 

12  4 

4 

4 

4 

4 

0 

0 

0 

0 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X2 
STATISTIC: PEARSON 
ALPHA:  .10 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.03, .07, .27, .63) 

.0564 

46 

2 

2 

2 

2 

2 

2 

2 

1 

1 

0 

(.04, .16, .16, .64) 

.0531 

36 

3 

3 

3 

3 

3 

1 

1 

1 

1 

0 

(.04, .06,. 36,. 54) 

.0441 

29 

2 

2 

2 

2 

2 

2 

2 

2 

2 

0 

(.05, .05, .45, .45) 

.0400 

27 

2 

2 

2 

2 

2 

2 

2 

2 

2 

0 

(.06, .14, .24, .56) 

.0361 

<  8 

4 

4 

4 

4 

4 

4 

3 

3 

3 

1 

(.08, .12, .32, .48) 

.0259 

<  8 

4 

4 

4 

4 

4 

4 

4 

3 

2 

2 

(.10, .10, .40,. 40) 

.0225 

9 

4 

4 

4 

4 

4 

4 

4 

2 

2 

2 

(.09, .21, .21, .49) 

.0216 

13 

4 

4 

4 

4 

3 

3 

3 

3 

1 

0 

(.12, .18, .28, .42) 

.0129 

21  * 

4 

4 

3 

3 

3 

2 

2 

1 

0 

0 

(.15, .15, .35, .35) 

.0100 

22 

4 

4 

4 

2 

2 

2 

2 

0 

0 

0 

(.16, .24, .24, .36) 

.0051 

22 

4 

4 

4 

3 

3 

1 

1 

0 

0 

0 

(.20,. 20,. 30,. .30) 

.0025 

23 

4 

4 

4 

4 

2 

2 

0 

0 

0 

0 

(.25, .25, .25, .25) 

.0000 

23 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0 

(.01, .09, .09, .81) 

.1056 

>  9b 

3 

3 

1 

1 

1 

1 

1 

1 

1 

1 

(.02, .08, .18, .72) 

.0769 

>  96 

2 

2 

2 

1 

1 

1 

1 

1 

1 

0 

TABLE:  2X3 
STATISTIC: PEARSON 
ALPHA:  .10 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

(.03,. 03,. 24,. 07,. 07,. 56) 

.0360 

30 

5 

5 

5 

4 

T 

4 

T 

T 

2 

2 

(.01, .03, .06, .09, .27, .54) 

.0351 

17 

6 

5 

5 

5 

5 

5 

4 

4 

4 

2 

( . 02 , . 04 ,.14, .08, .16, .56) 

.0334 

14 

6 

6 

6 

5 

5 

5 

5 

5 

4 

2 

(.05,. 05,. 40,. 05,. 05,. 40) 

.0272 

32 

4 

4 

4 

4 

4 

4 

4 

4 

4 

0 

(.02,. 04,. 04,. 18,. 36,. 36) 

.0214 

36 

4 

4 

4 

4 

3 

3 

3 

3 

3 

1 

(.02,. 08,. 10,. 08,. 32,. 40) 

.0198 

<12 

6 

6 

6 

6 

6 

6 

5 

4 

4 

3 

(.04,. 08,. 28,. 06,. 12,. 42) 

.0190 

30 

5 

5 

4 

4 

4 

4 

4 

3 

2 

0 

(.03, .09, .18, .07, .21, .42) 

.0167 

<12 

6 

6 

6 

6 

6 

5 

5 

5 

3 

2 

(.04, .06, .10, .16, .24, .40) 

.0153 

<12 

6 

6 

6 

6 

6 

6 

5 

5 

4 

2 

(.06, .06, .18, .14, .14, .42) 

.0148 

22 

6 

5 

5 

5 

5 

5 

5 

2 

2 

0 

( . 06 , . 06 , . 08 , . 24 , . 24 , . 32 ) 

.0108 

<12 

6 

6 

6 

6 

6 

6 

6 

5 

3 

3 

(.05, .15, .30, .05, .15, .30) 

.0106 

24 

6 

6 

6 

4 

4 

4 

4 

2 

2 

0 

(.10, .10, .30, .10, .10, .30) 

.0089 

<12 

6 

6 

6 

6 

6 

6 

6 

4 

4 

0 

(.04, .16, .20, .06, .24, .30) 

.0086 

<12 

6 

6 

6 

6 

6 

6 

6 

5 

3 

2 

(.06, .12, .12, .14, .28, .28) 

.0070 

<12 

6 

6 

6 

6 

6 

6 

6 

4 

4 

1 

(.08, .12, .20, .12, .18, .30) 

.0052 

<12 

6 

6 

6 

6 

6 

6 

6 

5 

3 

1 

(.10, .20, .20, .10, .20, .20) 

.0022 

<12 

6 

6 

6 

6 

6 

6 

6 

6 

2 

0 

(.12, .12, .16, .18, .18, .24) 

.0017 

<12 

6 

6 

6 

6 

6 

6 

6 

6 

3 

0 

(.17, .17, .17, .17, .17, .17) 

.0000 

<12 

6 

6 

6 

6 

6 

6 

6 

6 

0 

0 

(.01, .01, .08, .09, .09, .72) 
(.02, .02, .06, .18, .18,  .54) 


0624  >  96 

0324  >  96 


5532222222 

3333322220 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


TABLE:  2X4 
STATISTIC: PEARSON 
ALPHA:  .10 


(.01,. 
(.03,, 
(.01,, 
(.02,. 
(.01,. 
(.05,, 
(.02,, 
(.02,. 
( .04, . 
( *03, . 
(.02,. 
(.03,. 
(.05,. 
(.04,, 
(.04,, 
(•05,, 
(.06,. 
(.04,, 
(.10,, 
(.06,. 
(.13,. 


P  VECTOR 

MU  2 

N(MIN) 

,.01, .07, .09, .09, .09, .63) 

.0377 

37 

,.03, .21, .07, .07, .07, .49) 

.0221 

40 

,.03, .05, .09, .09, .27, .45) 

.0213 

21 

,.04, .12,. 08 , . 08 , . 16 , . 48) 

.0201 

16 

,.02, .05, .09, .18, .18, .45) 

.0192 

22 

,.05, .351.05, .05, .05, .35) 

.0169 

47 

,.08, .08, .08, .08, .32, .32) 

.0133 

<16 

, . 02, . 04, .18, .18, . 18, .36) 

.0131 

50 

, . 08 , .24, .06, . 06 ,.12,. 36 ) 

.0117 

40 

,.09, .15, .07, .07, .21, .35) 

.0105 

24 

,.06, .08, .08, .16, .24, .32) 

.0099 

<16 

,.06, .15, .07, .14, .14, .35) 

.0090 

18 

,.15, .25, .05, .05, .15, .25) 

.0069 

35 

, . 16, . 16, .06, .06, . 24, . 24) 

.0065 

16 

,.06, .06, .16, .16, .24, .24) 

.0065 

<16 

,.10, .25, .05, .10, .10, .25) 

.0056 

19 

, . 06 ,.12, .14, .14, .14, .28) 

.0047 

16 

,.12, .16, .06, .12, .18, .24) 

.0039 

24 

,.10, .20, .10, .10, .10, .20) 

.0019 

<16 

,.12, .12, .12, .12, .18, .18) 

.0013 

<16 

,.13, .13, .13, .13, .13, .13) 

.0000 

<16 

NUMBER  < 
VALUES 
10  9  B  7 

tttt 

7  7  6  6 

8  7  7  7 
8  8  8  7 
8  7  7  7 
6  6  6  6 
8  8  8  8 
7  4  4  4 

7  6  6  6 

8  8  7  7 
8  8  8  8 
8  8  8  8 
8  8  6  6 
8  8  8  8 
8  8  8  8 
8  8  8  8 
8  8  8  8 
8  8  8  8 
8  8  8  8 
8  8  8  8 
8  8  8  8 


OF  EXPECTED 
LESS  THAN 

6  5  4  3  2  1 

7  7  7  4  3  3 

6  6  6  6  3  0 

7  6  6  6  6  3 
7  7  7  7  6  3 

7  7  7  5  5  3 
6  6  6  6  0  0 

8  6  6  6  6  2 
4  4  4  4  3  0 

6  6  5  4  2  0 

7  6  6  5  4  2 

8  7  7  6  5  3 

7  7  7  7  4  1 
6  4  4  4  4  0 

8  8  8  6  4  4 
8  8  8  6  4  4 
8  8  6  6  6  2 
8  8  7  7  4  3 
8  7  6  5  3  1 
8  8  8  6  6  0 
8  8  8  8  6  0 
8  8  8  8  0  0 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


TABLE:  3X3 
STATISTIC: PEARSON 
ALPHA:  .10 


NUMBER 


EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.01,. 01,. 08,. 01,. 01,. 08,. 08,. 08,. 64) 

.0361 

48 

8 

8 

8 

8 

8 

8 

8 

4 

4 

4 

(.01,. 03,. 06,. 01,. 03,. 06,. 08,. 24,. 48) 

.0214 

31 

8 

8 

8 

7 

7 

7 

7 

7 

6 

4 

(.01,. 02,. 07,. 02,. 04,. 14,. 07,. 14,. 49) 

.0201 

26 

8 

8 

8 

8 

8 

8 

8 

6 

6 

3 

( . 02, . 02, . 06, . 02 , .02 , . 06, .16, .16,. 48 ) 

.0199 

20 

9 

8 

8 

8 

8 

8 

8 

6 

6 

4 

(.02,. 04,. 04,. 02,. 04,. 04,. 16,. 32,. 32) 

.0141 

50 

7 

7 

6 

6 

6 

6 

6 

6 

2 

0 

(.01,. 04,. OS,. 02,. 08,. 10,. 07,. 28,. 35) 

.0129 

<18 

9 

9 

9 

9 

8 

7 

7 

7 

7 

4 

( .  01 , .  03, .  06 , .  03, .  09 , .  18 , .  06 , .  18 , .  36  ) 

.0112 

<18 

9 

9 

9 

9 

8 

8 

8 

6 

6 

3 

(.02,. 03,. 05,. 04,. 06,. 10,. 14,. 21,. 35) 

.0105 

<18 

9 

9 

9 

9 

8 

8 

8 

7 

6 

4 

(.02,. 02,. 06,. 06,. 06,. 18,. 12,. 12,. 36) 

.0101 

<18 

9 

9 

9 

9 

8 

8 

8 

7 

5 

2 

(.04,. 04,. 12,. 04,. 04,. 12,. 12,. 12,. 36) 

.0092 

18 

9 

9 

9 

9 

8 

8 

8 

8 

4 

4 

(.03,. 03,. 04,. 06,. 06,. 08,. 21,. 21,. 28) 

.0081 

<18 

9 

9 

9 

9 

9 

8 

8 

6 

6 

3 

(.01,. 04,. 05,. 04,. 16,. 20,. 05,. 20,. 25) 

.0073 

<18 

9 

9 

9 

9 

9 

9 

8 

6 

5 

5 

(.02,. 04,. 04,. 06,. 12,. 12,. 12,. 24,. 24) 

.0061 

<18 

9 

9 

9 

9 

9 

9 

7 

7 

4 

3 

(.02,. 03,. 05,. 08,. 12,. 20,. 10,. 15,. 25) 

.0054 

<18 

9 

9 

9 

9 

9 

9 

8 

7 

5 

3 

(.04,. 08,. 08,. 04,. 08,. 08,. 12,. 24,. 24) 

.0053 

<18 

9 

9 

9 

9 

9 

9 

7 

7 

6 

2 

(.04,. 06,. 10,. 06,. 09,. 15,. 10,. 15,. 25) 

.0037 

<18 

9 

9 

9 

9 

9 

9 

8 

8 

6 

1 

(.03,. 03,. 04,. 12,. 12,. 16,. 15,. 15,. 20) 

.0035 

<18 

9 

9 

9 

9 

9 

9 

9 

8 

3 

3 

( . 04 , . 08 , . 08 , . 08 ,.16, .16,. 08 ,.16, .16) 

.0021 

<18 

9 

9 

9 

9 

9 

9 

9 

9 

5 

1 

(.06,. 06,. 08,. 09,. 09,. 12,. 15,. 15,. 20) 

.0020 

<18 

9 

9 

9 

9 

9 

9 

9 

8 

5 

0 

(.09,. 09,. 12,. 09,. 09,. 12,. 12,. 12,. 16) 

.0005 

<18 

9 

9 

9 

9 

9 

9 

9 

9 

4 

0 

(.11,. 11,. 11,. 11,. 11,. 11,. 11,. 11,. 11) 

.0000 

<18 

9 

9 

9 

9 

9 

9 

9 

9 

0 

0 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X2 
STATISTIC: PEARSON 
ALPHA:  .05 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.03, .07, .27, .63) 

.0564 

52 

2 

2 

2 

2 

2 

2 

2 

1 

1 

0 

(.04, .16, .16, .64) 

.0531 

38 

3 

3 

3 

3 

1 

1 

1 

1 

1 

0 

(.04, .06, .36, .54) 

.0441 

48 

2 

2 

2 

2 

2 

2 

2 

2 

1 

0 

(.05, .05, .45, .45) 

.0400 

44 

2 

2 

2 

2 

2 

2 

2 

2 

0 

0 

( .06, • 14, .24, . 56) 

.0361 

16 

4 

4 

3 

3 

3 

3 

3 

2 

1 

1 

(.08, .12, .32, .48) 

.0259 

18 

4 

4 

3 

3 

3 

2 

2 

2 

1 

0 

(.10,. 10, .40, .40) 

.0225 

17 

4 

4 

4 

4 

2 

2 

2 

2 

2 

0 

(.09, .21, .21, .49) 

.0216 

<  8 

4 

4 

4 

4 

4 

4 

4 

3 

3 

1 

(.12, .18, .28, .42) 

.0129 

<  8 

4 

4 

4 

4 

4 

4 

4 

3 

3 

1 

(.15, .15, .35, .35) 

.0100 

<  8 

4 

4 

4 

4 

4 

4 

4 

4 

2 

0 

(.16, .24, .24, .36) 

.0051 

17 

4 

4 

4 

4 

3 

3 

1 

1 

0 

0 

(.20,. 20,. 30,. 30) 

.0025 

18 

4 

4 

4 

4 

4 

2 

2 

0 

0 

0 

(.25, .25, .25, .25) 

.0000 

18 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

(.01, .09,. 09, .81) 

.1056 

> 

96 

3 

3  111111 

1  1 

(.02, .08, .18, .72) 

.0769 

> 

96 

2 

2  2  11111 

1  0 

314 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X3 
STATISTIC: PEARSON 
ALPHA:  .05 

NUMBER  OF  EXPECTED 


P  VECTOR  MU 2  N(MIN)  VALUES  LESS  THAN 


10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

(.01, .01, .08, .09, .09, .72) 

.0624 

62 

5 

5 

5 

5 

5 

3 

2 

2 

2 

2 

(.03, .03, .24, .07, .07, .56) 

.0360 

26 

5 

5 

5 

5 

4 

4 

4 

4 

4 

2 

(.01, .03, .06, .09, .27, .54) 

.0351 

33 

5 

5 

4 

4 

4 

4 

4 

4 

3 

2 

(.02, .04, .14, .08, .16, .56) 

.0334 

<12 

6 

6 

6 

6 

5 

5 

5 

5 

5 

3 

(.02, .02, .06, .18, .18, .54) 

.0324 

25 

5 

5 

5 

5 

5 

5 

3 

3 

3 

2 

(.05, .05, .40, .05, .05, .40) 

.0272 

45 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

(.02, .04, .04, .18, .36, .36) 

.0214 

48 

4 

4 

3 

3 

3 

3 

3 

3 

3 

1 

(.02, .08, .10, .08, .32, .40) 

.0198 

<12 

6 

6 

6 

6 

6 

6 

5 

4 

4 

3 

(.04, .08, .28, .06, .12, .42) 

.0190 

40 

4 

4 

4 

4 

4 

4 

3 

2 

1 

0 

(.03, .09, .18, .07, .21, .42) 

.0167 

<12 

6 

6 

6 

6 

6 

5 

5 

5 

3 

2 

(.04, .06, .10, .16, .24, .40) 

.0153 

19 

6 

6 

6 

5 

5 

5 

4 

3 

3 

1 

(.06, .06, .18, .14, .14, .42) 

.0148 

20 

6 

6 

5 

5 

5 

5 

5 

4 

2 

0 

(.06, .06, .08, .24, .24, .32) 

.0108 

25 

6 

6 

5 

5 

3 

3 

3 

3 

2 

0 

(.05, .15, .30, .05, .15, .30) 

.0106 

26 

6 

6 

6 

4 

4 

4 

4 

2 

2 

0 

(.10, .10, .30, .10, .10, .30) 

.0089 

18 

6 

6 

6 

6 

6 

4 

4 

4 

4 

0 

(.04, .16, .20, .06, .24, .30) 

.0086 

18 

6 

6 

6 

6 

6 

5 

4 

3 

2 

1 

(.06, .12, .12, .14, .28, .28) 

.0070 

<12 

6 

6 

6 

6 

6 

6 

6 

4 

4 

1 

(.08, .12, .20, .12, .18, .30) 

.0052 

<12 

6 

6 

6 

6 

6 

6 

6 

5 

3 

1 

(.10, .20, .20, .10, .20, .20) 

.U022 

<12 

6 

6 

6 

6 

6 

6 

6 

6 

2 

0 

(.12, .12, .16, .18, .18, .24) 

.0017 

<12 

6 

6 

6 

6 

6 

6 

6 

6 

3 

0 

(.17, .17, .17, .17, .17, .17) 

.0000 

<12 

6 

6 

6 

6 

6 

6 

6 

6 

0 

0 

5 


TABLE:  2X4 
STATISTIC: PEARSON 
ALPHA:  .05 


(.01, 

(.03, 

(.01, 

(.02, 

(.01, 

(.05, 

(.02, 

(.02, 

(.04, 

(.03, 

(.02, 

(.03, 

(.05, 

(.04, 

(.04, 

(-05, 

(.06, 

(.04, 

(.10, 

(.08, 

(.13, 


P  VECTOR 

MU2 

N(MIN) 

,.07, .09, .09, .09, .63) 

.0377 

76 

,.21, .07, .07, .07, .49) 

.0221 

28 

>,. 05,  .09,  .09,  .27,  .45) 

.0213 

24 

,.12, .08, .08, .16, .48) 

.0201 

<16 

:,. 05, .09, .18, .18, .45) 

.0192 

23 

i,. 35, .05, .05, .05, .35) 

.0169 

50 

I,.  08,  .08,  .08,  .32,  .32) 

.0133 

<16 

:,. 04, .18, .18, .18, .36) 

.0131 

50 

. , .  24 , .  06 , .  06 , .  12, .  36) 

.0117 

44 

,.15, .07, .07, .21, .35) 

.0105 

16 

:,. 08, .08, .16, .24, .32) 

.0099 

<16 

,.15, .07, .14, .14, .35) 

.0090 

18 

,.25, .05, .05, .15, .25) 

.0069 

38 

,.16,. Oo , . 06 , . 24 , • 24 ) 

.0065 

70 

, .06, •  16, . 16, . 24, .24) 

.0065 

<16 

,.25, .05, .10, .10, .25) 

.0056 

28 

,.12, .14, .14, .14, .28) 

.0047 

<16 

,.16, .06, .12, .18, .24) 

.0039 

28 

i,. 20, .10, .10, .10, .20) 

.0019 

21 

,.12, .12, .12, .18, .18) 

.0013 

30 

,.13, .13, .13, .13, .13) 

.0000 

<16 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 
VALUES  LESS  THAN 
10  987654321 
7  777433333 
7777766663 
7777666643 
8887777  /  63 
7777775543 
6666666600 
8888866662 
7444444430 
6666655420 
8888877652 
8888877653 
8888777741 
8666644440 
4444442200 
8888888644 
8886666620 
8888887743 
8888765320 
8888886600 
8888866200 
8888888800 


6 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  3X3 
STATISTIC: PEARSON 
ALPHA:  .05 

NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

(. 01,. 01,. 08,. 01,. 01,. 08,. 08,. 08,. 64) 

.0361 

60 

8 

8 

8 

8 

8 

8 

4 

4 

4 

4 

(.01,. 03,. 06,. 01,. 03,. 06,. 08,. 24,. 48) 

.0214 

54 

7 

7 

7 

7 

7 

7 

6 

4 

4 

2 

(.01,. 02,. 07,. 02,. 04,. 14,. 07,. 14, .49) 

.0201 

49 

8 

8 

8 

8 

6 

6 

6 

4 

4 

3 

(.02,. 02,. 06,. 02,. 02,. 06,. 16,. 16,. 48) 

.0199 

27 

8 

8 

8 

8 

8 

8 

6 

6 

6 

4 

(.02,. 04,. 04,. 02,. 04,. 04,. 16,. 32,. 32) 

.0141 

32 

7 

7 

7 

7 

7 

6 

6 

6 

6 

2 

(.01,. 04,. 05,. 02,. 08,. 10,. 07,. 28,. 35) 

.0129 

<18 

9 

9 

9 

9 

8 

7 

7 

7 

7 

4 

(.01,. 03,. 06,. 03,. 09,. 18,. 06,. 18,. 36) 

.0112 

<18 

9 

9 

9 

9 

8 

8 

8 

6 

6 

3 

(.02,. 03,. 05,. 04,. 06,. 10,. 14,. 21,. 35) 

.0105 

22 

9 

9 

9 

8 

8 

8 

7 

6 

5 

3 

(.02,. 02,. 06,. 06,. 06,. 18,. 12,. 12,. 36) 

.0101 

<18 

9 

9 

9 

9 

8 

8 

8 

7 

5 

2 

(.04,. 04,. 12,. 04,. 04,. 12,. 12,. 12,. 36) 

.0092 

<18 

9 

9 

9 

9 

8 

8 

8 

8 

4 

4 

( .03, . 03 , . 04 , . 06 , . 06 , . 08 ,.21, .21, .28) 

.0081 

30 

9 

9 

8 

8 

6 

6 

6 

6 

5 

2 

( . 01 , . 04 , . 05 , . 04 , . 16 , . 20 , . 05 , .20, .25) 

.0073 

<18 

9 

9 

9 

9 

9 

9 

8 

6 

5 

5 

(.02,. 04,. 04,. 06,. 12,. 12,. 12,. 24,. 24) 

.0061 

<18 

9 

9 

9 

9 

9 

9 

7 

7 

4 

3 

(.02,. 03,. 05,. 08,. 12,. 20,. 10,. 15,. 25) 

.0054 

<13 

9 

9 

9 

9 

9 

9 

8 

7 

5 

3 

( . 04 , . 08 , . 08, . 04 , • 08 , . 08 , . 12 , . 24 , . 24 ) 

.0053 

20 

9 

9 

9 

9 

9 

9 

7 

7 

6 

2 

(.04,. 06,. 10,. 06,. 09,. 15,. 10,. 15,. 25) 

.0037 

<18 

9 

9 

9 

9 

9 

9 

8 

8 

6 

1 

(.03,. 03,. 04,. 12,. 12,. 16,. 15,. 15,. 20) 

.0035 

20 

9 

9 

9 

9 

9 

9 

8 

5 

3 

3 

( . 04 , . 08 , . 08 , .08, . 16, . 16, .08, . 16, . 16) 

.0021 

23 

9 

9 

9 

9 

9 

9 

9 

5 

5 

1 

(.06,. 06,. 08,. 09,. 09,. 12,. 15,. 15,. 20) 

.0020 

<18 

9 

9 

9 

9 

9 

9 

9 

8 

5 

0 

(.09,. 09,. 12,. 09,. 09,. 12,. 12,. 12,. 16) 

.0005 

<18 

9 

9 

9 

9 

9 

9 

9 

9 

4 

0 

(.11,. 11,. 11,. 11,. 11,. 11,. 11,. 11,. 11) 

.0000 

18 

9 

9 

9 

9 

9 

9 

9 

9 

0 

0 

TABLE:  2X5 
STATISTIC: PEARSON 
ALPHA:  .05 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.01,. 01,. 01,. 01,. 06,. 09,. 09,. 09,. 09,. 54) 

.0228 

74 

9  9  9  9 

5 

5 

4 

4 

4 

4 

( .03, .03, .03, .03, . 18, .07, .07, .07, .07, .42) 

.0132 

58 

8  8  8  8 

8 

8 

4 

4 

4 

0 

(.01,. 01,. 01,. 03,. 04,. 09,. 09,. 09,. 27,. 36) 

.0130 

42 

8  8  8  8 

8 

8 

8 

5 

5 

3 

(.02,. 02,. 02,. 04,. 10,. 08,. 08,. 08,. 16,. 40) 

.0118 

55 

9  9  8  8 

8 

7 

4 

4 

3 

0 

(.05,. 05,. 05,. 05,. 30,. 05,. 05,. 05,. 05,. 30) 

.0100 

64 

8  8  8  8 

8 

8 

8 

0 

0 

0 

(.01,. 01,. 02,. 03,. 03,. 09,. 09,. 18,. 27,. 27) 

.0097 

44 

8  8  8  7 

7 

7 

7 

5 

5 

3 

(.02,. 02,. 04,. 04,. 08,. 08,. 08,. 16,. 16,. 32) 

.0077 

<20 

10101010 

9 

9 

9 

7 

7 

4 

(.04,. 04,. 04,. 08,. 20,. 06,. 06,. 06,. 12,. 30) 

.0066 

58 

8  8  8  8 

7 

7 

6 

3 

0 

0 

( .03, .03, .03, .09, .12, .07, .07, .07, .21, .28) 

.0062 

43 

9  8  8  8 

8 

7 

7 

3 

3 

0 

(.02,. 04,. 04,. 04,. 06,. 08,. 16,. 16,. 16,. 24) 

.0050 

50 

9  9  6  6 

6 

6 

5 

4 

1 

0 

(.05,. 05,. 05,. 15,. 20,. 05,. 05,. 05,. 15,. 20) 

.0040 

66 

8  6  6  6 

6 

6 

6 

0 

0 

0 

(.03,. 03,. 06,. 09,. 09,. 07,. 07,. 14,. 21,. 21) 

.0039 

40 

1010  8  8 

8 

7 

7 

5 

2 

0 

(.04,. 04,. 08,. 08,. 16,. 06,. 06,. 12,. 12,. 24) 

.0035 

31 

101010  9 

9 

9 

8 

6 

4 

0 

(.05,. 05,. 10,. 15,. 15,. 05,. 05,. 10,. 15,. 15) 

.0020 

27 

101010101010 

6 

6 

4 

0 

(.04,. 08,. 08,. 08,. 12,. 06,. 12,. 12,. 12,. 18) 

.0014 

24 

101010101010 

9 

9 

5 

1 

( . 10, . 10, . 10, . 10, . 10, . 10, . 10, . 10, . 10, . 10) 

.0000 

25 

1010101010101010 

0 

0 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X2 
STATISTIC: PEARSON 
ALPHA:  .01 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU  2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

(.02, .08, .18, .72) 

.0769 

46 

3 

3 

2 

2 

2 

2 

2 

1 

1 

1 

(.03, .07, .27, .63) 

.0564 

10 

4 

4 

4 

4 

3 

3 

3 

3 

2 

2 

(.04, .16, .16, .64) 

.0531 

21 

3 

3 

3 

3 

3 

3 

3 

1 

1 

1 

(.04, .06, .36, .54) 

.0441 

48 

2 

2 

2 

2 

2 

2 

2 

2 

1 

0 

(.05, .05, .45, .45) 

.0400 

56 

2 

2 

2 

2 

2 

2 

2 

2 

0 

0 

(.06, .14, .24, .56) 

.0361 

9 

4 

4 

4 

4 

4 

3 

3 

3 

2 

1 

(.08, .12, .32, .48) 

.0259 

<  8 

4  4  4  4  4  4  4  : 

J  2  i 

> 

(.10, .10, .40, .40) 

.0225 

16 

4 

4 

4 

4 

2 

2 

2 

2 

2 

0 

(.09, .21, .21, .49) 

.0216 

<  8 

4 

4 

4 

4 

4 

4 

4 

3 

3 

1 

(.12, .18, .28, .42) 

.0129 

<  8 

4 

4 

4 

4 

4 

4 

4 

3 

2 

1 

(.15, .15, .35, .35) 

.0100 

<  8 

4 

4 

4 

4 

4 

4 

4 

4 

2 

0 

(.16, .24, .24, .36) 

.0051 

<  8 

4 

4 

4 

4 

4 

4 

4 

4 

3 

0 

(.20, .20, .30, .30) 

.0025 

<  8 

4 

4 

4 

4 

4 

4 

4 

4 

2 

0 

(.25, .25, .25, .25) 

.0000 

<  8 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

(.01, .09, .09, .81) 


1056  >  96 


3  3  11111111 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 

TABLE:  2X3 
STATISTIC: PEARSON 
ALPHA:  .01 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.01, .03, .06, .09, .27, .54) 

.0351 

81 

4 

4 

4 

3 

3 

3 

2 

2 

1 

1 

(.02, .04, .14, .08, .16, .56) 

.0334 

30 

5 

5 

5 

5 

5 

5 

3 

3 

2 

1 

(.02, .02, .06, .18, .18, .54) 

.0324 

24 

5 

5 

5 

5 

5 

5 

3 

3 

3 

2 

(.05, .05, .40, .05, .05, .40) 

.0272 

61 

4 

4 

4 

4 

4 

4 

4 

0 

0 

0 

(.02, .04, .04, .18, .36, .36) 

.0214 

13 

6 

6 

6 

6 

6 

6 

4 

4 

3 

3 

(.02, .08, .10, .08, .32, .40) 

.0198 

<12 

6 

6 

6 

6 

6 

6 

5 

4 

4 

3 

(.04, .08, .28, .06, .12, .42) 

.0190 

30 

5 

5 

4 

4 

4 

4 

4 

3 

2 

0 

(.03, .09,. 18, .07, .21, .42) 

.0167 

<12 

6 

6 

6 

6 

6 

5 

5 

5 

3 

2 

(.04, .06, .10, .16, .24, .40) 

.0153 

<12 

6 

6 

6 

6 

6 

6 

5 

5 

4 

2 

( . 06 , . 06, .18, .14, .14,. 42 ) 

.0148 

<12 

6 

6 

6 

6 

6 

5 

5 

5 

4 

2 

(.06, .06, .08, .24, .24, .32) 

.0108 

13 

6 

6 

6 

6 

6 

6 

5 

3 

3 

2 

(.05, .15, .30, .05, .15, .30) 

.0106 

32 

6 

4 

4 

4 

4 

4 

2 

2 

2 

0 

(.10, .10, .30, .10, .10, .30) 

.0089 

24 

6 

6 

6 

4 

4 

4 

4 

4 

0 

0 

(.04,. 16,. 20,. 06,. 24,. 30) 

.0086 

21 

6 

6 

6 

6 

5 

4 

3 

2 

2 

1 

(.06, .12, .12, .14, .28, .28) 

.0070 

<12 

6 

6 

6 

6 

6 

6 

6 

4 

4 

1 

(.08, .12, .20, .12, .18, .30) 

.0052 

20 

6 

6 

6 

b 

5 

5 

4 

3 

1 

0 

(.10, .20, .20, .10, .20, .20) 

.0022 

22 

6 

6 

6 

6 

6 

6 

2 

2 

0 

0 

(.12, .12, .16, .18, .18, .24) 

.0017 

15 

6 

6 

6 

6 

6 

6 

6 

5 

2 

0 

(.17, .17, .17, .17, .17, .17) 

.0000 

15 

6 

6 

6 

6 

6 

6 

6 

6 

0 

0 

(.01, .01, .08, .09, .09, .72) 

.0624 

>  96 

5 

5 

3 

2 

2 

2 

2 

2 

2 

2 

(.03, .03, .24, .07, .07, .56) 

.0360 

>  96 

4 

4 

4 

4 

2 

2 

2 

0 

0 

0 

TABLE:  2X4 
STATISTIC : PEARSON 
ALPHA:  .01 


P  VECTOR 


(. 

.01, 

.01, 

.03, 

.05, 

.09, 

.09, 

•  27, 

.45) 

(. 

.02, 

.02, 

.04, 

.12, 

.08, 

.08, 

.16, 

.48) 

(. 

.01, 

.02, 

.02, 

.05, 

.09, 

.18, 

.18, 

.45) 

(< 

.05, 

.05, 

.05, 

•35, 

.05, 

.05, 

.05, 

.35) 

(. 

.02, 

.02, 

.08, 

.08, 

.08, 

.08, 

.32, 

.32) 

(. 

.02, 

.02, 

.02, 

.04, 

.18, 

.18, 

.18, 

.36) 

(. 

.04, 

.04, 

.08, 

.24, 

.06, 

.06, 

.12, 

.36) 

<■ 

.03, 

.03, 

.09, 

•15, 

.07, 

*.07, 

.21, 

.35) 

(. 

.02, 

.04, 

.06, 

.08, 

.08, 

.16, 

.24, 

.32) 

( 

.03, 

.06, 

.06, 

•15, 

.07, 

•14, 

.14, 

.35) 

< 

.05, 

.05, 

.15, 

.25, 

.05, 

.05, 

.15, 

.25) 

( 

.04, 

.04, 

.16, 

.16, 

.06, 

.06, 

.24, 

.24) 

( 

.04, 

.04, 

.06, 

.06, 

.16, 

.16, 

.24, 

.24) 

(. 

.05, 

.10, 

.10, 

.25, 

.05, 

.10, 

.10, 

.25) 

( 

.06, 

.06, 

.06, 

•12, 

•  14, 

.14, 

•14, 

.28) 

( 

.04, 

.08, 

.12, 

.16, 

.06, 

•12, 

.18, 

.24) 

( 

.10, 

.10, 

.10, 

.20, 

.10, 

.10, 

.10, 

.20) 

( 

.08, 

.08, 

.12, 

.12, 

.12, 

.12, 

.18, 

.18) 

( 

•13, 

•13, 

•13, 

.13, 

•  13, 

•13, 

•13, 

.13) 

( 

.01, 

.01, 

.01, 

.07, 

.09, 

.09, 

.09, 

.63) 

( 

.03, 

.03, 

.03, 

.21, 

.07, 

.07, 

.07, 

.49) 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0213 

94 

6 

6 

4 

4 

4 

4 

3 

3 

2 

2 

0201 

25 

7 

7 

7 

7 

7 

7 

6 

5 

3 

2 

0192 

80 

5 

5 

5 

4 

4 

4 

3 

3 

3 

1 

0169 

68 

6 

6 

6 

6 

6 

6 

6 

0 

0 

0 

0133 

32 

6 

6 

6 

6 

6 

6 

6 

6 

2 

2 

0131 

<16 

8 

8 

8 

8 

8 

7 

7 

7 

4 

4 

0117 

80 

6 

5 

5 

5 

4 

4 

2 

0 

0 

0 

0105 

20 

8 

8 

8 

7 

7 

7 

6 

5 

5 

2 

0099 

<16 

8 

8 

8 

8 

8 

7 

7 

6 

5 

3 

0090 

<16 

8 

8 

8 

8 

8 

7 

7 

7 

4 

3 

0069 

55 

6 

6 

4 

4 

4 

4 

4 

4 

0 

0 

0065 

55 

6 

6 

4 

4 

4 

4 

4 

2 

0 

0 

0065 

<16 

8 

8 

8 

8 

8 

8 

8 

6 

4 

4 

0056 

37 

8 

6 

6 

6 

6 

6 

6 

2 

2 

0 

0047 

<16 

8 

8 

8 

8 

8 

8 

7 

7 

4 

3 

0039 

40 

8 

7 

7 

6 

5 

5 

3 

2 

1 

0 

0019 

23 

8 

8 

8 

8 

8 

8 

6 

6 

0 

0 

0013 

30 

8 

8 

8 

8 

8 

6 

6 

2 

0 

0 

0000 

24 

8 

8 

8 

8 

8 

8 

8 

0 

0 

0 

0377 

> 

96 

7  7 

4 

4 

3 

3 

3 

3 

3 

3 

0221 

> 

96 

6  6 

6 

6 

3 

3 

3 

3 

0 

0 

TABLE:  3X3 
STATISTIC: PEARSON 
ALPHA:  .01 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


P  VECTOR 

MU  2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

( • 01 , » 03 , • 06 1 <01| • 03, . 06 , • OB , • 24 , • 48) 

.0214 

99 

7 

7 

7 

6 

6 

4 

4 

4 

2 

2 

(.01,. 02,. 07,. 02,. 04,. 14,. 07,. 14,. 49) 

.0201 

91 

6 

6 

6 

6 

4 

4 

4 

3 

3 

1 

(.02,. 02,. 06,. 02,. 02,. 06,. 16,. 16,. 4S) 

.0199 

79 

6 

6 

6 

6 

6 

6 

4 

4 

4 

0 

(.02,. 04,. 04,. 02,. 04,. 04,. 16,. 32,. 32) 

.0141 

<18 

9 

9 

9 

9 

9 

7 

7 

7 

6 

6 

(.01,. 04,. 05,. 02,. 06,. 10,. 07,. 28,. 35) 

.0129 

52 

7 

7 

7 

7 

7 

6 

5 

4 

2 

1 

(.01,. 03,. 06,. 03,. 09,. 18,. 06,. 18,. 36) 

.0112 

46 

8 

8 

6 

6 

6 

6 

5 

5 

3 

1 

(.02,. 03,. 05,. 04,. 06,. 10,. 14,. 21,. 35) 

.0105 

18 

9 

9 

9 

9 

8 

8 

8 

7 

6 

4 

(.02,. 02,. 06,. 06,. 06,. 18,. 12,. 12,. 36) 

.0101 

<18 

9 

9 

9 

9 

8 

8 

8 

7 

5 

2 

(.04,. 04,. 12,. 04,. 04,. 12,. 12,. 12,. 36) 

.0092 

<18 

9 

9 

9 

9 

8 

8 

8 

8 

4 

4 

(.03,. 03,. 04,. 06,. 06,. 08,. 21,. 21,. 28) 

.0081 

30 

9 

9 

8 

8 

6 

6 

6 

6 

5 

2 

(.01,. 04,. 05,. 04,. 16,. 20,. 05,. 20,. 25) 

.0073 

27 

9 

9 

9 

9 

8 

6 

5 

5 

5 

1 

(.02,. 04,. 04,. 06,. 12,. 12,. 12,. 24,. 24) 

.0061 

20 

9 

9 

9  9 

9 

9 

7 

7 

4 

3 

(.02,. 03,. 05,. 08,. 12,. 20,. 10,. 15,. 25) 

.0054 

<18 

9 

9 

9 

9 

9 

9 

8 

7 

5 

3 

(.04,. 08,. 08,. 04,. 08,. 08,. 12,. 24,. 24) 

.0053 

20 

9 

9 

9 

9 

9 

9 

7 

7 

6 

2 

(.04,. 06,. 10,. 06,. 09,. 15,. 10,. 15,. 25) 

.0037 

<18 

9 

9 

9 

9 

9 

9 

8 

8 

6 

1 

(.03,. 03,. 04,. 12,. 12,. 16,. 15,. 15,. 20) 

.0035 

<18 

9 

9 

9 

9 

9 

9 

9 

8 

3 

3 

(.04,. 08,. 08,. 08,. 16,. 16,. 08,. 16,. 16) 

.0021 

25 

9 

9 

9 

9 

9 

9 

5 

5 

1 

0 

(.06,. 06,. 08,. 09,. 09,. 12,. 15,. 15,. 20) 

.0020 

<18 

9 

9 

9 

9 

9 

9 

9 

8 

5 

0 

(.09,.  09,.  12,.  09,.  09,.  12,.  12,.  1-2,.  16) 

.0005 

22 

9 

9 

9 

9 

9 

9 

9 

8 

4 

0 

(.11,. 11,. 11,. 11,. 11,. 11,. 11,. 11,. 11) 

.0000 

<18 

9 

9 

9 

9 

9 

9 

9 

9 

0 

0 

(.01, .01,. 08 ,.01, .01,. 08 , . 08 , . 08 , . 64) 

.0361 

>108 

8 

8 

4 

4 

4 

4 

4 

4 

4 

0 

(.01,. 01,. 08,. 01,. 01,. 08,. 08,. 06,. 64) 

.0361 

>108 

8 

8 

4 

4 

4 

4 

4 

4 

4 

0 

TABLE:  2X5 
STATISTIC : PEARSON 
ALPHA:  .01 


(.03, .03, 
(.01, .01, 
(.02, .02, 
(.05, .05, 
(.01, .01, 
(.02,. 02, 
( .04, .04, 
(.03, .03, 
(.02,. 04, 
(.05, .05, 
(.03, .03, 
(.04,. 04, 
(.05, .05, 
(.04, .08, 
(.10, .10, 


.03,. 03, 
.01, .03, 
.02, .04, 
.05, .05, 
.02,. 03, 
.04,. 04, 
. 04 , . 08 , 
.03, .09, 
.04,. 04, 
.05, .15, 
.06,. 09, 
.08,. 08, 
.10, .15, 
.  08 , . 08, 
.10,. 10, 


VECTOR 

MU2 

N(MI1 

18, .07, .07, .07, .07, .42) 

.0132 

46 

04, .09, .09, .09, .27, .36) 

.0130 

85 

10, .08, .08, .08, .16, .40) 

.0118 

55 

30,. 05,. 05,. 05,. 05,. 30) 

.0100 

58 

03, .09, .09, .18, .27, .27) 

.0097 

54 

08, .08, .08, .16, .16, .32) 

.0077 

<20 

20, .06, .06, .06, .12, .30) 

.0066 

44 

12, .07, .07, .07, .21, .28) 

.0062 

56 

06, .08, .16, .16, .16, .24) 

.0050 

60 

20, .05, .05, .05, .15, .20) 

.0040 

49 

09, .07, .07, .14, .21, .21) 

.0039 

33 

16, .06, .06, .12, .12, .24) 

.0035 

54 

15, .05, .05, .10, .15, .15) 

.0020 

32 

12, .06, .12, .12, .12, .18) 

.0014 

28 

10, .10, .10, .10, .10, .10) 

.0000 

26 

06, .09, .09, .09, .09, .54) 

.0228 

>100 

06, .09, .09, .09, .09, .54) 

.0228 

>100 

CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 
VALUES  LESS  THAN 
10  987654321 
9  9  8  8  8  8  8  4  4  0 
8885555433 
9988874430 
8888888800 
8777775552 
10101010  999774 
9988877630 
8888766330 
9666665410 
10  886666600 
10101010  887732 
9988664200 
101010101010  6440 
1010101010  99520 
1010101010101010  0  0 


9555444440 

9555444440 


323 


TABLE:  S2X2X2 
STATISTIC: PEARSON 
ALPHA:  .10 

P  VECTOR 

( . 050 , . 050 , . 050 , . 050 , . 200 , . 200 , . 200 , . 200 ) 
(.048, .072, .072, . 108, . 112, . lbtt, . 168, .252) 
(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 
( .064, .096, .096, . 144, .096, . 144, . 144, .216) 
( . 100, . 100, . 100, . 100, . 150, . 150, . 150, . 150) 
(.125,. 125,. 125,. 125,. 125,. 125,. 125,. 125) 

(.008, .012, .032, .048, .072, . 108, .288, .432) 
(.009,. 021,. 021,. 049,. 081,. 189,. 189,. 441) 
( .016, .024, .064, .096, .064, .096, .256, .384) 
(.015,. 015,. 035,. 035,. 135,. 135,. 315,. 315) 
(.018,. 042,. 042,. 098,. 072,. 168,. 168,. 392) 
(.016,. 024,. 024,. 036,. 144,. 216,. 216,. 324) 
(.025,. 025,. 025,. 025,. 225,. 225,. 225,. 225) 
( .030, .030, .070, .070, . 120, . 120, .280, .280) 
(.027,. 063, .063, . 147, .063, . 147, . 147, .343) 
(.032,. 048,. 048,. 072,. 128,. 192,. 192,. 288) 
(.045,. 045,. 105,. 105,. 105,. 105,. 245,. 245) 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0056 

93 

4 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0040 

56 

7 

5 

5 

5 

3 

3 

1 

1 

0 

0 

0025 

44 

8 

8 

8 

4 

4 

4 

4 

0 

0 

0 

0020 

56 

7 

7 

4 

4 

4 

1 

1 

0 

0 

0 

0006 

47 

8 

8 

8 

4 

4 

4 

0 

0 

0 

0 

0000 

38 

8 

8 

8 

8 

8 

8 

0 

0 

0 

0 

0206 

>104 

5 

5 

5 

4 

4 

4 

3 

2 

2 

1 

0189 

>104 

5 

5 

4 

4 

4 

3 

3 

3 

1 

1 

0144 

>104 

6 

4 

4 

4 

2 

2 

2 

2 

1 

0 

0141 

>104 

4 

4 

4 

4 

4 

4 

4 

2 

2 

0 

0130 

>104 

4 

4 

4 

3 

3 

3 

1 

1 

1 

0 

0121 

>104 

4 

4 

4 

4 

4 

4 

4 

3 

1 

0 

0100 

>104 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

0090 

>104 

4 

4 

4 

2 

2 

2 

2 

0 

0 

0 

0088 

>104 

4 

4 

4 

4 

1 

1 

1 

1 

0 

0 

0074 

>104 

4 

4 

4 

3 

3 

3 

1 

0 

0 

0 

0054 

>104 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

TABLE:  2X2X2 
STATISTIC : PEaRSON 
ALPHA:  .10 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


P  VECTOR 

(.003,. 007,. 027,. 063,. 027,. 063,. 243,. 567) 
(.004,. 016,. 016,. 064,. 036,. 144,. 144,. 576) 
( . 005 , . 005 , . 045 , . 045 , . 045 , . 045 , . 405 , . 405 ) 
( .008, .032, .032, . 128, .032, . 128, . 128, .512) 
(.008,. 012,. 032,. 048,. 072,. 108,. 288,. 432) 
(.009,. 021,. 021,. 049,. 081,. 189,. 189,. 441) 
( .016, .024, .064, .096, .064, .096, .256, .384) 
( .015, .015, .035, .035, . 135, . 135, .315, .315) 
( .018, .042, .042, .098, .072, . 168, . 168, .392) 
(.016,. 024,. 024,. 036,. 144,. 216,. 216,. 324) 
( . 025 , . 025 , . 025 , . 025 , . 225 , . 225 , . 225 , . 225 ) 
(.030,. 030,. 070,. 070,. 120,. 120,. 280,. 280) 
(.027,. 063,. 063,. 147,. 063,. 147,. 147,. 343) 
(.032,. 048,. 048,. 072,. 128,. 192,. 192,. 288) 
( . 050 , . 050 , . 050 , . 050 , . 200 , . 200 , . 200 , . 200 ) 
(.045,. 045,. 105,. 105,. 105,. 105,. 245,. 245) 
(.048,. 072,. 072,. 108,. 112,. 168,. 168,. 252) 
(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 
(.064,. 096,. 096,. 144,. 096,. 144,. 144,. 216) 
(.100,. 100,. 100,. 100,. 150,. 150,. 150,. 150) 
(.125,. 125,. 125,. 125,. 125,. 125,. 125,. 125) 

(.001,. 009,. 009,. 081,. 009,. 081,. 081,. 729) 


NUMBER  OF  EXPECTED 


MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0331 

63 

6 

6 

6 

6 

6 

6 

_6 

4 

4 

2 

0318 

37 

7 

7 

7 

7 

7 

5 

5 

5 

4 

3 

0264 

56 

6 

6 

6 

6 

6 

6 

6 

6 

2 

2 

0237 

19 

8 

7 

7 

7 

7 

7 

7 

7 

4 

4 

0206 

27 

7 

7 

7 

6 

6 

6 

6 

6 

5 

3 

0189 

<16 

8 

8 

8 

7 

7 

7 

7 

5 

5 

4 

0144 

<16 

8 

8 

8 

8 

7 

7 

6 

6 

6 

2 

0141 

<16 

8 

8 

8 

8 

8 

6 

6 

6 

4 

4 

0130  • 

<16 

8 

8 

8 

8 

7 

7 

7 

7 

5 

3 

0121 

37 

7 

7 

7 

5 

5 

4 

4 

4 

4 

3 

0100 

43 

8 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0090 

<16 

8 

8 

8 

8 

8 

8 

6 

6 

6 

2 

0088 

<16 

8 

8 

8 

8 

8 

7 

7 

7 

4 

1 

0074 

<16 

8 

8 

8 

8 

8 

8 

7 

5 

4 

3 

0056 

<16 

8 

8 

8 

8 

8 

8 

8 

4 

4 

4 

0054 

<16 

8 

8 

8 

8 

8 

8 

8 

6 

6 

2 

0040 

<16 

8 

8 

8 

8 

8 

8 

7 

7 

5 

1 

0025 

<16 

8 

8 

8 

8 

8 

8 

8 

8 

4 

0 

0020' 

<16 

8 

8 

8 

8 

8 

8 

8 

7 

4 

0 

0006 

<16 

8 

8 

8 

8 

8 

8 

8 

8 

4 

0 

0000 

<16 

8 

8 

8 

8 

8 

8 

8 

8 

0 

0 

0533 

>  96 

7 

7 

7 

4 

4 

4 

4 

4 

4 

4 

TABLE:  2X2X3 
STATISTIC : PEARSON 
ALPHA:  .10 


(.002,. 002,. 
(.005,. 005,. 
(.004,. 008,. 
(.010, .010,. 
(.003,. 009,. 
(.004,. 006,. 
(.009,. 018,. 
(.006, .018,. 
(.010,. 010,. 
(.004,. 016,. 
(.016,. 016,. 
(.008,. 012,. 
(.006, .012,. 
(.020,. 020,. 
(.008,. 032,. 
(.012,. 012,. 
(.012,. 024,. 
(.020,. 040,. 
(.012,. 036,. 
(.018, .027,. 
(.015,. 060,. 
(.024,. 024,. 
(.032, .032,. 
(.025,. 075,. 
(.024, .048,. 
(.040, .060,. 
(.045, .045,. 
(.050,. 100,. 
(.083,. 083,. 


P  VECTOR 


,.054, .162, 
,.360, .045, 
,.112,. 064, 
,.320,. 040, 
, . 162, .063, 
,.090,. 144, 
,.147, .049, 
,.144,. 056, 
,.270,. 090, 
,.180, .054, 
,.192,. 036, 
,.080,. 128, 
,.108,. 126, 
,.240,. 080, 
,.160,. 048, 
,.144,. 162, 
,.096,. 112, 
,  .210, .030, 
,.168,. 042, 
,.105,. 098, 
,.175,. 035, 
,.128,. 144, 
,.144,. 072, 
,.150,. 025, 
,.112,. 084, 
,.150,. 060, 
,.140,. 105, 
,.100,. 050, 
,.083,. 083, 


.162, .486) 
.045,. 360) 
.128,. 448) 
.040,. 320) 
.189,. 378) 
.216, .360) 
.098,. 343) 
.168,. 336) 
.090,. 270) 
.216, .270) 
.036,. 288) 
.192,. 320) 
.252,. 252) 
.080,. 240) 
.192,. 240) 
.162,. 216) 
.224,. 224) 
.060, .210) 
.126,. 252) 
.147, .245) 
.140, .175) 
.144,. 192) 
.072,. 216) 
.075,. 150) 
.168,. 168) 
.090,. 150) 
.105,. 140) 
.100, .100) 
.083,. 083) 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OP  EXPECTED 
VALUES  LESS  THAN 
10  987654321 
9  9  9  9  9  7  1  i  i  3 
101010101010  6444 
111111111110  8874 
121212101010101010  8 
1110  99998764 
11101010  999874 
12121111111111  984 
12121111111110  985 
1212121210101010  6  6 
121212121110  9887 
12111111  998886 
12121111111110  975 
12121212101010  976 
121212121210101010  4 
121212121110  9884 
12121212121111  866 
1212121212101010  7  4 
12101010  888862 
12121212111110  974 
1212121212111110  7  2 
12121212121210  884 
12121212121211  865 
12121212121111  982 
12121212121212  884 
1212121212121010  7  2 
1212121212121210  6  2 
1212121212121210  6  0 
1212121212121212  4  0 
1212121212121212  0  0 


(.001,. 001,. 008,. 009,. 009,. 072,. 009,. 009,. 072,. 081,. 081,. 648) 
(  .002, .004, .014, .008, .016, .056, .018, .036, . 126, .072, . 144, .504) 


KU2 

N(MIN) 

,0177 

96 

,0156 

96 

,0139 

43 

,0118 

<24 

0113 

50 

0107 

42 

0082 

<24 

0082 

<24 

0081 

<24 

0080 

<24 

0079 

32 

0077 

25 

0073 

24 

0055 

<24 

0054 

27 

0051 

<24 

0049 

<24 

0048 

44 

0046 

<24 

0037 

<24 

0032 

<24 

0031 

<24 

0030 

<24 

0026 

<24 

0021 

<24 

0013 

<24 

0013 

<24 

0006 

<24 

0000 

<24 

0300 

>  96 

0181 

>  96 

llllll  9777777 
9999877663 


TABLE:  S2X2X2 
STATISTIC: PEARSON 
ALPHA:  .05 

P  VECTOR 

( .025, .025, .025, .025, .225, .225, .225, .225) 
( .030, .030, .070, .070, . 120, .120, .280, .280) 
(.027,. 063,. 063,. 147,. 063,. 147,. 147,. 343) 
(.032,. 048,. 048,. 072,. 128,. 192,. 192,. 288) 
( .050, .050, .050, .050, .200, .200, .200, .200) 
( .045, .045, . 105, . 105, . 105, . 105, .245 , .245) 
(.048,. 072,. 072,. 108,. 112,. 168,. 168,. 252) 
(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 
(.064,. 096,. 096,. 144,. 096,. 144,. 144,. 216) 
( . 100, . 100, . 100, . 100, . 150, . 150, . 150, . 150) 
(.125,. 125,. 125,. 125,. 125,. 125,. 125,. 125) 

(.008,. 012,. 032,. 048,. 072,. 108,. 288,. 432) 
(.009,. 021,. 021,. 049,. 081,. 189,. 189,. 441) 
(.016, .024, .064, .096, .064, .096, .256, .384) 
(.015,. 015,. 035,. 035,. 135,. 135,. 315,. 315) 
( .018, .042, .042, .098, .072, . 168, . 168, .392) 
(.016,. 024,. 024,. 036,. 144,. 216,. 216,. 324) 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

.0100 

85 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

.0090 

60 

6 

6 

6 

4 

4 

4 

2 

2 

2 

0 

.0088 

36 

7 

7 

7 

7 

7 

4 

4 

4 

1 

1 

.0074 

48 

7 

5 

5 

5 

4 

4 

4 

3 

1 

0 

.0056 

35 

8 

8 

8 

4 

4 

4 

4 

4 

4 

0 

.0054 

42 

6 

6 

6 

6 

6 

6 

2 

2 

2 

0 

.0040 

57 

7 

5 

5 

5 

3 

3 

1 

1 

0 

0 

.0025 

36 

8 

8 

8 

8 

4 

4 

4 

4 

0 

0 

.0020 

49 

7 

7 

7 

4 

4 

4 

1 

0 

0 

0 

.0006 

42 

8 

8 

8 

8 

4 

4 

0 

0 

0 

0 

.0000 

40 

8 

8 

8 

8 

8 

0 

0 

0 

0 

0 

.0206 

>104 

5 

5 

5 

4 

4 

4 

3 

2 

2 

1 

.0189 

>104 

5 

5 

4 

4 

4 

3 

3 

3 

1 

1 

.0144 

>104 

6 

4 

4 

4 

2 

2 

2 

2 

1 

0 

.0141 

>104 

4 

4 

4 

4 

4 

4 

4 

2 

2 

0 

.0130 

>104 

4 

4 

4 

3 

3 

3 

1 

1 

1 

0 

.0121 

>104 

4 

4 

4 

4 

4 

4 

4 

3 

1 

0 

‘AtV-l*  A.’  V 


I 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


TABLE:  2X2X2 
STATISTIC: PEARSON 
ALPHA:  .05 


NUMBER 


EXPECTED 


P  VECTOR 

MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

(.004,. 016,. 016,. 064,. 036,. 144,. 144,. 576) 

.0318 

88 

5 

5 

5 

5 

5 

4 

4 

3 

3 

1 

(.005,. 005,. 045,. 045,. 045,. 045,. 405,. 405) 

.0264 

73 

6 

6 

6 

6 

6 

6 

6 

2 

2 

2 

(.008,. 032,. 032,. 128,. 032,. 128,. 128,. 512) 

.0237 

35 

7 

7 

7 

7 

7 

7 

4 

4 

4 

1 

( .008, .012, .032, .048, .072, . 108, .288, .432) 

.0206 

31 

7 

7 

6 

6 

6 

6 

6 

5 

4 

3 

(.009,. 021,. 021,. 049,. 081,. 189,. 189,. 441) 

.0189 

23 

7 

7 

7 

7 

7 

7 

5 

5 

5 

3 

( .016, .024, .064, .096, .064, .096, .256, .384) 

.0144 

<16 

8 

8 

8 

8 

7 

7 

6 

6 

6 

2 

(.015,. 015,. 035,. 035,. 135,. 135,. 315,. 315) 

.0141 

<16 

8 

8 

8 

8 

8 

6 

6 

6 

4 

4 

( .018, .042, .042, .098, .072, . 168, . 168, .392) 

.0130 

<16 

8 

8 

8 

8 

7 

7 

7 

7 

5 

3 

(.016,. 024,. 024,. 036,. 144,. 216,. 216,. 324) 

.0121 

34 

7 

7 

7 

5 

5 

5 

4 

4 

4 

3 

( .025, .025, .025, .025, .225, .225, .225, .225) 

.0100 

56 

4 

4 

4 

4 

4 

4 

4 

4 

4 

0 

( .030, .030, .070, .070, . 120, . 120, .280, . 280) 

.0090 

<16 

8 

8 

8 

8 

8 

8 

6 

6 

6 

2 

(.027,. 063,. 063,. 147,. 063,. 147,. 147,. 343) 

.0088 

<16 

8 

8 

8 

8 

8 

7 

7 

7 

4 

1 

(.032,. 048,. 0'48,. 072,.  128,.  192,.  192,. 288) 

.0074 

<16 

8 

8 

8 

8 

8 

8 

7 

5 

4 

3 

( . 050 , . 050 , . 050 , . 050 , . 200 , . 200 , . 200 , . 200 ) 

.0056 

<16 

8 

8 

8 

8 

8 

8 

8 

4 

4 

4 

(.045,. 045,. 105,. 105,. 105,. 105,. 245,. 245) 

.0054 

<16 

8 

8 

8 

8 

8 

8 

8 

6 

6 

2 

(.048,. 072,. 072,. 108,. 112,. 168,. 168,. 252) 

.0040 

<16 

8 

8 

8 

8 

8 

8 

7 

7 

5 

1 

(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 

.0025 

<16 

8 

8 

8 

8 

8 

8 

8 

8 

4 

0 

(.064,. 096,. 096,. 144,. 096,. 144,. 144,. 216) 

.0020 

<16 

8 

8 

8 

8 

8 

8 

8 

7 

4 

0 

(.100,. 100,. 100,. 100,. 150,. 150,. 150,. 150) 

.0006 

<16 

8 

8 

8 

8 

8 

8 

8 

8 

4 

0 

(.125,. 125,. 125,. 125,. 125,. 125,. 125,. 125) 

.0000 

<16 

8 

8 

8 

8 

8 

8 

8 

8 

0 

0 

(.001,. 009,. 009,. 081,. 009,. 081,. 081,. 729) 

.0533 

>  96 

7 

7 

7 

4 

4 

4 

4 

4 

4 

4 

(.003,. 007,. 027,. 063,. 027,. 063,. 243,. 567) 

.0331 

>  96 

6 

6 

6 

6 

4 

4 

4 

4 

2 

2 

TABLE:  2X2X3 
STATISTIC: PEARSON 
ALPHA:  .03 

?  VECTOR 

( .004, .008, .028, .016, .032, . 112, .016, .032, .  112, .064, 
(. 010,. 010,. 080,. 010,. 010,. 080,. 040,. 040,. 320,. 040, 
( .003, .009, .01S, .007, .021, .042, .027, .081, . 162, .063, 
( .004, .006, .010, .016, .024, .040, .036, .034, .090, . 144, 
( .009, .018, .063, .021 , .042, . 147, .021, .042, . 147 , .049, 
(.006,. 018,. 036,. 014,. 042,. 084,. 024,. 072,. 144,. 056, 
(.010,. 010,. 030, .010,. 010,. 030,. 090,. 090,. 270,. 090, 
( .004, .016, .020, .006, .024, .030, .036, . 144 , . 160, .054, 
( .016, .016, . 128, .024, .024, . 192, .024, .024 , . 192, .036, 
( .008, .012, .020, .032, .048, .080, .032, .048, .080, .128, 
(.006,. 012,. 012,. 014,. 028,. 028,. 054,. 108,. 108,. 126, 
( .020, .020, .060, .020, .020, .060, .080, .080, .240, .080, 
( .008, .032, .040, .012, .048, .060, .032, . 128, . 160, .048, 
(.012,. 012,. 016,. 018,. 018,. 024,. 108,. 108,. 144,. 162, 
( .012, .024, .024, .028, .056, .056, .048, .096, .096, . 112, 
( .020, . 040, . 140, .020, . 040, . 140, .030, .060, .210, .030, 
( .012, . 036, .072, .018, .054, . 108, .028, .084, . 168, .042, 
( .018, .027, .045, .042, .063, . 105, .042, .063 , . 105, .098, 
(.015,. 060,. 075,. 015,. 060,. 075,. 035,. 140,. 175,. 035, 
( .024, .024, .032, .036, .036, .048, .096, .096, . 128, .144, 
( .032, .032, .096, .048, .048, . 144, .048, .048, . 144, .072, 
( .025, .075, . 150, .025, .075, . 150, .025, .075, . 150, .025 , 
( .024, .048, .048, .036, .072, .072, .056, . 112, . 112, .084, 
( . 040, . 060, . 100, . 040 , . 060, . 100 , . 060 , . 090 , . 150 , . 060 , 
( .045, .045, .060, .045, .045, .060, . 105, . 105, . 140, . 105, 
( .050, . 100, . 100, .050, . 100, . 100, .050, . 100, . 100 , . 050, 
(.083,. 083,. 083,. 083, .083, .083,. 083,. 083,. 083,. 083, 

(.001,. 001,. 008,. 009,. 009,. 072,. 009,. 009,. 072,. 081, 
( .002, .004, .014, .008, .016, .056, .018, .036, . 126, .072, 
(.002,. 002,. 006,. 018,. 018,. 054,. 018,. 018,. 054,. 162, 
(.005,. 005,. 040,. 005, .005, .040, .045, .045, .360, .045, 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OP  EXPECTED 


MU2 

N(MIN) 

VALUES  LESS  THAN 
10  987654321 

128,. 448) 

.0139 

76 

1110  888877 

4 

2 

040,. 320) 

.0118 

31 

1210101010101010 

8 

4 

189,. 378) 

.0113 

63 

99999887 

6 

3 

216,. 360) 

.0107 

71 

99998887 

5 

3 

098,. 343) 

.0082 

<24 

12121111111111  9 

8 

4 

168,. 336) 

.0082 

36 

1111111110  998 

6 

4 

090,. 270) 

.0081 

26 

1212121010101010 

6 

6 

216,. 270) 

.0080 

49 

1010  988888 

7 

4 

036,. 288) 

.0079 

55 

99988888 

8 

2 

192, .320) 

.0077 

26 

12121111111110  9 

7 

5 

252,. 252) 

.0073 

25 

12121212101010  9 

7 

6 

080,. 240) 

.0055 

<24 

121212121210101010  4 

192,. 240) 

.0054 

<24 

12121212121110  8  8 

5 

162,. 216) 

.0051 

<24 

12121212121111  8 

6 

6 

224,. 224) 

.0049 

<24 

1212121212101010 

7 

4 

060,. 210) 

.0048 

41 

1212101010  888 

6 

2 

126,. 252) 

.0046 

34 

121211111110  9  8 

6 

3 

147,. 245) 

.0037 

<24 

1212121212111110 

7 

2 

140,. 175) 

.0032 

<24 

12121212121210  8 

8 

4 

144,. 192) 

.0031 

46 

12121111  9866 

5 

0 

072, .216) 

.0030 

<24 

12121212121111  9 

8 

2 

075,. 150) 

.0026 

<24 

12121212121212  8 

8 

4 

168,. 168) 

.0021 

29 

12121212121210  8 

5 

1 

090,. 150) 

.0013 

<24 

1212121212121210 

6 

2 

105,. 140) 

.0013 

<24 

1212121212121210 

6 

0 

100,. 100) 

.0006 

29 

1212121212121212 

4 

0 

083,. 083) 

.0000 

<24 

1212121212121212 

0 

0 

081,. 648) 

.0300 

>  96 

111111  97777 

7 

7 

144,. 504) 

.0181 

>  96 

99998776 

6 

3 

162,. 486) 

.0177 

>  96 

99999777 

7 

3 

045,. 360) 

.0156 

>  96 

101010101010  6  4 

4 

4 

TABLE:  S2X2X2 
STATISTIC: PEARSON 
ALPHA:  .01 

P  VECTOR 

(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 
(.064,. 096,. 096,. 144,. 096,. 144,. 144,. 216) 
( . 100, . 100, . 100, . 100, . 150, . 150, . 150, .150) 
(.125,. 125,. 125,. 125,. 125,. 125,. 125,. 125) 

(.008,. 012,. 032,. 048,. 072,. 108,. 288,. 432) 
(.009,. 021,. 021,. 049,. 081,. 189,. 189,. 441) 
(.016, .02 ., .064, .096, .064, .096, .256, . 384) 
(.015,. 015,. 035,. 035,. 135,. 135,. 315,. 315) 
( .018, .042, .042, .098, .072, . 168, . 168, .392) 
(.016,. 024,. 024,. 036,. 144,. 216,. 216,. 324) 
( .025, .025, .025, .025, .225, .225, .225, .225) 
( .030, .030, .070, .070, . 120, . 120 ,. 280, . 280) 
(.027,. 063,. 063,. 147,. 063,. 147,. 147,. 343) 
(.032,. 048,. 048,. 072,. 128,. 192,. 192,. 288) 
( . 030 , . 050 , . 050 , . 050 , . 200 , . 200 , . 200 , . 200 ) 
(.045,. 045,. 105,. 105,. 105,. 105,. 245,. 245) 
(.048,. 072,. 072,. 108,. 112,. 168,. 168,. 252) 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


MU2 

N(MIN) 

VALUES 

LESS 

THAN 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

.0025 

72 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0 

.0020 

56 

7 

7 

4 

4 

4 

1 

1 

0 

0 

0 

.0006 

40 

8 

8 

8 

8 

4 

4 

0 

0 

0 

0 

.0000 

37 

8 

8 

8 

8 

8 

8 

0 

0 

0 

0 

.0206 

>104 

5 

5 

5 

4 

4 

4 

3 

2 

2 

1 

.0189 

>104 

5 

5 

4 

4 

4 

3 

3 

3 

1 

1 

.0144 

>104 

6 

4 

4 

4 

2 

2 

2 

2 

1 

0 

.0141 

>104 

4 

4 

4 

4 

4 

4 

4 

2 

2 

0 

.0130 

>104 

4 

4 

4 

3 

3 

3 

1 

1 

1 

0 

.0121 

>104 

4 

4 

4 

4 

4 

4 

4 

3 

1 

0 

.0100 

>104 

4 

4 

4 

4 

4 

4 

4 

4 

0 

0 

.0090 

>104 

4 

4 

4 

2 

2 

2 

2 

0 

0 

0 

.0088 

>104 

4 

4 

4 

4 

1 

1 

1 

1 

0 

0 

.0074 

>104 

4 

4 

4 

3 

3 

3 

1 

0 

0 

0 

.0056 

>104 

4 

4 

4 

4 

4 

0 

0 

0 

0 

0 

.0054 

>104 

2 

2 

2 

2 

2 

2 

0 

0 

0 

0 

.0040 

>104 

3 

3 

3 

1 

1 

1 

0 

0 

0 

0 

m 

I 

m 

Mil 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


TABLE:  2X2X2 
STATISTIC: PEARSON 
ALPHA:  .01 


P  VECTOR 

MU2 

N(HIN) 

VALUES 
10  9  8  7 

LESS 

6  5  4 

THAN 

3  2  1 

(.008,. 012,. 032,. 048,. 072,. 108,. 288,. 432) 

.0206 

63 

6 

6 

6 

6 

5 

5 

4 

3 

2 

2 

(.009,. 021,. 021,. 049,. 081,. 189,. 189,. 441) 

.0189 

46 

7 

7 

5 

5 

5 

5 

5 

4 

3 

3 

(.016,. 024,. 064,. 096,. 064,. 096,. 256,. 384) 

.0144 

41 

6 

6 

6 

6 

6 

6 

6 

4 

2 

2 

(.015,. 015,. 035,. 035,. 135,. 135,. 315,. 315) 

.0141 

<16 

8 

8 

8 

8 

8 

6 

6 

6 

4 

4 

(.018,. 042,. 042,. 098,. 072,. 168,. 168,. 392) 

.0130 

42 

7 

7 

7 

5 

5 

5 

4 

3 

3 

1 

(.016,. 024,. 024,. 036,. 144,. 216,. 216,. 324) 

.0121 

20 

8 

8 

8 

8 

7 

7 

5 

5 

4 

4 

( . 025 , . 025 , . 025 , . 025 , . 225 , . 225 , . 225 , . 225 ) 

.0100 

17 

8 

8 

8 

8 

8 

8 

8 

4 

4 

4 

(.030,. 030,. 070,. 070,. 120,. 120,. 280,. 280) 

.0090 

<16 

8 

8 

8 

8 

8 

8 

6 

6 

6 

2 

( .027, .063, .063, . 147, .063, . 147, .147, .343) 

.0088 

<16 

8 

8 

8 

8 

8 

7 

7 

7 

4 

1 

(.032,. 048,. 048,. 072,. 128,. 192,. 192,. 288) 

.0074 

<16 

8 

8 

8 

8 

8 

8 

7 

5 

4 

3 

( . 050 , . 050 , . 050 , . 050 , . 200 , . 200 , . 200 , . 200 ) 

.0056 

<16 

8 

8 

8 

8 

8 

8 

8 

4 

4 

4 

(.045,. 045,. 105,. 105,. 105,. 105,. 245,. 245) 

.0054 

<16 

8 

8 

8 

8 

8 

8 

8 

6 

6 

2 

(.048,. 072,. 072,. 108,. 112,. 168,. 168,. 252) 

.0040 

<16 

8 

8 

8 

8 

8 

8 

7 

7 

5 

1 

(.075,. 075,. 075,. 075,. 175,. 175,. 175,. 175) 

.0025 

<16 

8 

8 

8 

8 

8 

8 

8 

8 

4 

0 

(.064,. 096,. 096,. 144,. 096,. 144,. 144,. 216) 

.0020 

<16 

8 

8 

8 

8 

8 

8 

8 

7 

4 

0 

( . 100, . 100, . 100, . 100, . 150, . 150, . 150, . 150) 

.0006 

<16 

8 

8 

8 

8 

8 

8 

8 

8 

4 

0 

( . 125, . 125, . 125, . 125, . 125, . 125, . 125, . 125) 

.0000 

<16 

8 

8 

8 

8 

8 

8 

8 

8 

0 

0 

(.001,. 009,. 009,. 081,. 009,. 081,. 081,. 729) 

.0533 

>  96 

7 

7 

7 

4 

4 

4 

4 

4 

4 

4 

(.003,. 007,. 027,. 063,. 027,. 063,. 243,. 567) 

.0331 

>  96 

6 

6 

6 

6 

4 

4 

4 

4 

2 

2 

(.004,. 016,. 016,. 064,. 036,. 144,. 144,. 576) 

.0318 

>  96 

5 

5 

5 

5 

4 

4 

4 

3 

3 

1 

(.005,. 005,. 045,. 045,. 045,. 045,. 405,. 405) 

.0264 

>  96 

6 

6 

6 

6 

6 

6 

2 

2 

2 

2 

(.008,. 032,. 032,. 128,. 032,. 128,. 128,. 512) 

.0237 

>  96 

4 

4 

4 

4 

4 

4 

4 

1 

1 

1 

r+or 


Pm*  _  •-  A  %  ■  •  . 
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v 

IS 


TABLE:  2X2X3 
STATISTIC : PEARSON 
ALPHA:  .01 


CRITICAL  EXPECTED  VALUE  DISTRIBUTION 


NUMBER  OF  EXPECTED 


P  VECTOR 


(.010, 

(.009, 

(.006, 

(.010, 

(.016, 

(.008, 

(.006, 

(.020, 

(.008, 

(.012, 

(.012, 

(.020, 

(.012, 

(.016, 

(.015, 

(.024, 

(.032, 

(.025, 

(.024, 

(.040, 

(.045, 

(.050. 

(.083, 

(.001, 

(.002, 

(.002, 

(.005, 

(.004, 

(.003, 

(.004, 

(.004, 

(.004, 


.010, .080, . 
.018, .063, . 
.018, .036,. 
.010,. 030,. 
.016,. 128,. 
.012, .020,. 
.012, .012, . 
.020, .060, . 
.032,. 040,. 
.012, .016, . 
.024,. 024,. 
.040, . 140, . 
.036, .072,. 
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